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We consider the problem of subspace estimation in situations where the number of available snapshots 
and the observation dimension are comparable in magnitude. In this context, traditional subspace methods 
tend to fail because the eigenvectors of the sample correlation matrix are heavily biased with respect to the 
true ones. It has recently been suggested that this situation (where the sample size is small compared to the 
observation dimension) can be very accurately modeled by considering the asymptotic regime where the 
observation dimension M and the number of snapshots N converge to +oo at the same rate. Using large 
C*~) . random matrix theory results, it can be shown that traditional subspace estimates are not consistent in this 

(N 

£f) • asymptotic regime. Furthermore, new consistent subspace estimate can be proposed, which outperform 

C^j ' the standard subspace methods for realistic values of M and N. The work carried out so far in this area 

has always been based on the assumption that the observations are random, independent and identically 
distributed in the time domain. The goal of this paper is to propose new consistent subspace estimators 
for the case where the source signals are modelled as unknown deterministic signals. In practice, this 
allows to use the proposed approach regardless of the statistical properties of the source signals. In 



order to construct the proposed estimators, new technical results concerning the almost sure location of 
the eigenvalues of sample covariance matrices of Information plus Noise complex Gaussian models are 
established. These results are believed to be of independent interest. 
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Index Terms 

Subspace-based estimation, random matrix theory, information plus noise model, limit eigenvalue 
distribution. 

Notation: Matrix (resp. vectors) quantities are denoted by boldfaced capital (resp. lower case) letters. 
The N x N identity matrix is denoted as I at. Trace and spectral norm will be denoted Tr [•] and ||-|| 
respectively, and [-] T and [-] H represent the transpose and the conjugate transpose. For a set U, we denote 
by Int(W) and dlA its interior and boundary respectively. Given a complex number z, Re (z) and Im (z) 
denote its real and imaginary parts respectively, (•)* stands for complex conjugation and i denotes the 
imaginary unit. The upper complex half plane is denoted by C+, i.e C+ = {z G C : lm(z) > 0}, 
and equivalently C_ will denote the lower complex half plane. Similarly, R + and R_ represent the set 
of all positive real numbers and the set of all negative real numbers respectively. We will also write 
R* = R\ {0} and C* = C\{0}. For a given contour C on the complex plane, Indc(£) will denote the 
index of the contour with respect to a point £ G C. The support of a particular function <fi will be denoted 
as supp (4>), and C£°(R, R) will represent the set of compactly supported real- valued smooth functions 
defined on R. 

I. Introduction 

Subspace estimation methods have been widely proposed in multiple applications of communications 
and signal processing, such as direction of arrival (Do A) estimation (2, beamforming O, channel 
identification Q, waveform estimation 01, and many other general parametric estimation problems based 
on multivariate observations @. In general terms, these algorithms are applicable to the situation where a 
number of parameters needs to be extracted from a set of multivariate observations, which are composed 
of a noise part, with full-rank empirical correlation matrix, plus a signal contribution that has low-rank 
empirical correlation matrix. By exploiting the inherent orthogonality between the signal subspace (i.e. 
the subspace spanned by the columns of the signal empirical correlation matrix) and the noise subspace, 
one can try to extract the original parameters from the set of noisy observations. In general terms, the 
resulting estimators are computationally much more affordable and hence are generally preferred over 
other estimators such as those based in the Maximum Likelihood (ML) principle, which generally perform 
better but unfortunately involve an exhaustive search in a multi-dimensional parametric space. 

In order to formulate a generic subspace estimator, one must first infer the eigenvectors of the correlation 
matrix of the observation. This is generally difficult, because the correlation matrix of the multivariate 
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observation is generally unknown. In consequence, classical subspace estimation methods make use of 
the empirical correlation matrix, and approximate the eigenvectors of the true correlation matrix as the 
eigenvectors of the sample estimate. This procedure is clearly optimal when the number of observations 
(denoted by N) tends to infinity while the observation dimension (denoted by M) remains constant. 
Indeed, under certain ergodicity assumptions, when N — > oo for a fixed M, the sample correlation 
matrix of the observation converges almost surely to the true one, and consequently when N » M 
the sample eigenvectors (i.e. the eigenvectors of the sample correlation matrix) tend to be very good 
representations of the true ones. In practical applications, however, the number of available observations 
(N) and the observation dimension (M) are comparable in magnitude, which leads to strong discrepancies 
between the sample eigenvectors and the true ones. This originates what is usually referred to as the 
breakdown effect of subspace-based techniques Q. 

The fact that sample eigenvectors are not the best estimators of the true ones has been known for 
decades, although the study of valid alternatives to the classical estimators has been limited by the fact 
that investigations basically concentrated on the regime where N » M. However, it has been recently 
suggested Q that finite sample size situations (whereby N and M are comparable in magnitude) can 
be better examined by investigating the asymptotic regime in which M and N converge to +00 at the 
same rate, i.e. M,N — > +00, whereas = converges towards a strictly positive constant. Using 
Large Random Matrix Theory (LRMT) results, it was shown in that traditional subspace estimators 
are asymptotically biased in this asymptotic regime. Furthermore, consistent estimators for this regime 
can be found, which outperform the traditional ones for realistical values of M and N. In this context, 
LRMT can be very useful (1) to characterize how the sample eigenvectors differ from the true ones in 
a scenario where M and N are comparable in magnitude and (2) to derive alternative estimators of the 
eigenvectors that converge, not only when N — > +00 for a fixed M, but also when M, N — > +00 at 
the same rate. This was more extensively demonstrated in [8] and [9], which respectively considered the 
characterization of the sample eigenvectors when M, N — > +00 at the same rate, and proposed alternative 
consistent estimators for these quantities in the new asymptotic regime. 

Unfortunately, the work in (H and @ cannot be applied to the signal plus noise model considered 
here, unless the observations are random multivariate quantities that are Gaussian, independent and 
identically distributed in the time domain. In practice, however, there are multiple applications in which 
the observation does not present this structure, and is better modelled as a deterministic component 
(corresponding to the signal part) plus some additive noise, that is generally Gaussian distributed. This 
model is usually referred to as the "information plus noise model" in the LRMT literature |[T0l , as opposed 
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to the more classical "sample covariance matrix model" ifTTl . which was the one used in 0, (H, O. 
The main objective of this paper is to propose improved subspace estimators for the information plus 
noise model, which will represent the case where the source signals are as non-observable deterministic 
sequences. In order to obtain these estimators, new mathematical results related to the almost sure location 
of the eigenvalues of the empirical covariance matrix of a Gaussian information plus noise model are 
derived. These results are believed to be of independent interest. 

The rest of the paper is organized as follows. Section |II] introduces the information plus noise model 
associated with the specific application addressed here: the determination of multiple directions of arrival 
(DoA) using an array of antennas. The main objectives of the paper in mathematical terms are also 
formulated. Section |lll] provides some general facts related to the convergence of the eigenvalues of the 
empirical correlation matrix for the information plus noise model. It is further explained in Section |IV] 
that the eigenvalues of the sample correlation matrix tend to concentrate around some clusters when both 
M, N — > +00 at the same rate. A very simple description of the position of these asymptotic eigenvalue 
clusters is also provided. It is in particular shown that each cluster is associated with a set of consecutive 
eigenvalues of true covariance matrix of the observation. Section |V] presents an intermediate result that 
has its own interest. In brief, it is shown that, for sufficiently large M, N, with probability one no 
eigenvalues of the sample correlation matrix will be located outside the asymptotic eigenvalue clusters. 
Furthermore, the number of sample eigenvalues that are located in each of these clusters is directly related 
to the dimensionality of the corresponding eigenspace of the true covariance matrix. In order to focus 
on the applicative context of the paper, this claim is proved for the cluster associated with the noise 
subspace, but it can be extended easily to the other clusters. This fact generalizes the results derived in 
|[T2l and |[T3l in the context of source signals independent identically distributed in the time domain. 
In contrast with Ifl2l and |[T3l , the results presented in this paper, inspired by the approach developed 
in |[T4l . are only valid in the complex Gaussian case. The above mentioned results are then used in 
Section |VT] in order to derive an estimator of the localization function of the subspace estimate that is 
consistent not only when N — > +00 for fixed M, but also when M, N — > +00 at the same rate. Section 
I VII I provides some numerical examples that illustrate the effectiveness of the proposed estimators. Finally 
Section [Villi concludes the paper. Most of the technical derivations have been relegated to the appendices. 

The results of this paper have been partly presented in the short conference paper lfl5l . 
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II. Problem statement 

In order to motivate and illustrate the signal model that is used in this paper, we consider the following 
DoA estimation problem. Assume that K narrow band deterministic source signals {sk)k=\,...,K are 
received by an antenna array of M elements, K < M. The corresponding M dimensional observation 
signal y n (at discrete time n) can be mathematically described as 

y n = As n + v n 

where A = [a(6>i), . . . , sl(6k)] is an M x K matrix that contains the steering vectors of the K sources, 
s n is a K x 1 column vector containing the transmitted signals from the K sources at time instant n, 
and where v„ is an additive Gaussian white noise component with zero mean and covariance matrix 
E [v n v^] = <t 2 Im- We assume that y n is available from n = 1 to n = N, and that M < N, or 
equivalently that c/y = 77 is strictly less than 1. It is possible to generalize our results to the situation 
where cn > 1, although the presentation of the corresponding results would however complicate the 
developments of the present paper. 

We denote by Yat = [yi, • • • , Yn] the M x N observation matrix, which can be readily written as 

Y N = AS N + V N (1) 

where Sjv = [si, . . . , s^v] and Vjy = [vi, . . . , Vjv]- From this matrix, we can define the empirical spatial 
correlation matrix of the observation as Rjy = -^YtvY^, whereas the empirical spatial correlation matrix 
associated with the noiseless observation will take the form ASjvS^A^. It is worth pointing out here 
that, since the number of signals is assumed to be lower than the number of antennas (K < M), the 
steering matrix A will always be a tall matrix and therefore the empirical spatial correlation matrix of 
the noiseless observation will never be full rank. In other words, the minimum eigenvalue of the matrix 
-^AStvS^A^ will always be zero and will have multiplicity equal to M — K. 
In order to simplify the notation in the subsequent exposition, we define the matrices Sat, Bjv, Wjv 

as 

Y N „ AS^ ,„ Vat 



'N VN VN 

so that (Q]) can be equivalently formulated as 



B N = -^r, W N = -± (2) 



B N + W N (3) 



where S^r is the (normalized) matrix of observations, B^r is a deterministic matrix containing the signals 
contribution and W^r is a complex Gaussian white noise matrix with i.i.d. entries that have zero mean and 
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variance a 2 /N. We denote by FLat the orthogonal projection matrix on the "noise subspace", which in our 
context is defined as the orthogonal complement of the column space of matrix A. In the following, we 
assume that the empirical correlation matrix of Sat defined by -^SatS^ is full rank. Therefore, the noise 
subspace coincides with the kernel of the empirical correlation matrix of the noiseless signal, namely 
B N B%. 

Let { 7t | denote the eigenvalues of the empirical correlation matrix of the signal component, 

I J k=l,...,M 

namely BatB^, arranged in increasing order and let { e if^ } denote the corresponding unit norm 

eigenvectors. We note in particular that 7^ = . . . = "y^j_ K = while the remaining eigenvalues are 
strictly positive and that II at = Yl!k=i K e k^ ( e *^) ' "^ ne su ^ s P ace met hod for the determination of 
the K directions of arrival (commonly referred to as MUSIC algorithm) is based on the observation that 
the angles {#fc} fc=1 K coincide with the K solutions of the equation &(9) h Hn8l(9) = 0. In order to 
be able to use this last observation, it is in practice necessary to estimate the function a.(9) H UN&(6) 
(usually referred to as the "localization function") for each 9 G [— tt, 7r], or more generically to estimate 
the quantity 

m (b) = b H U N b 

for each deterministic M-dimensional vector b. 

If N — > +00 while M is fixed, the empirical correlation matrix of the observations Rjy = SatS^ of 
Yat converges towards the matrix Rat = B^B^ + ct 2 Im in the sense that 

\\R N - (B N B% + a 2 I M )\\ ^ a.s. (4) 

where a.s. represents the almost sure convergence. We will denote by |a^| the eigenvalues of 

Rat arranged in increasing order and by jej^ j the corresponding eigenvectors. The convergence 

result in (0]) implies that for each 9, r}^ ad (a(9)) — t]n(&(9)) — > a.s. where ff™ d (a(9)) is the traditional 
estimator of the localization function defined as 

M—K 

«a(0)) = £ a fl (fl)f (ef) a(6»). (5) 

fc=i 

In practice, predictions provided by the asymptotic regime corresponding to letting N — > +00 for fixed 
M are reliable only if N is much larger than M. However, this assumption may be quite restrictive in a 
number of important application contexts. If M and N are comparable in magnitude, then the asymptotic 
regime described by letting M, N — > +00 in such a way that cat = % converges towards a non zero 
constant appears to be more relevant. In this regime, the behavior of various classical estimates are 
more complicated, and have to be studied carefully. In particular, it can be shown that ff^ih) — r/Ar(b) 
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does not converge to when M, N — > +00 , which implies that the standard MUSIC estimates are 
not consistent under this new asymptotic regime. The purpose of this paper is to introduce an improved 
subspace estimate f/^ ew (b) of 77jv(b) for each deterministic vector b. The main feature of ff}f w (h) is 
to be consistent if M,N — > +00 in such a way that cn = 77- converges towards a non zero constant 
value. In order to achieve this, we will heavily rely on results related to the asymptotic behavior of the 
eigenvalue distribution of the empirical correlation matrix Rat- It is however useful to mention that it is 
not established that 

sup m ew (eL(e))-r, N (ei(e))\ (6) 

almost surely, a useful, but stronger property. We feel that the proof of (O would need mathematical 
technics different from those which are used in the present paper. 

III. Properties of the asymptotic eigenvalue distribution of matrix Tl N 

In this section, we will review some of the important properties related to the asymptotic behavior of 
the eigenvalue distribution of the empirical correlation matrix when M, N — > +00 in such a way 
that cat = 4£ converges towards a non zero constant, which will be denoted as c*. This implies that the 
observation dimension M in principle depends on N, and should be denoted M(N). We will however 
drop this dependence on N in order to simplify the exposition. Whenever it is clear from the context, 
we will also drop the dependence on the number of snapshots N in matrices E^r, Btv, R/v, eigenvalues 
,. . . , %m an d 7i v • ■ >1m > as weu as eigenvectors. 

Remark 1. From now on, N — > 00 will implicitly denote the limit as both M, N — > +00 such that j£ 
converges towards a non zero constant c*, where it is assumed that < c* < 1. 



Remark 2. All results that are presented in this paper are equally valid regardless of the behavior of 
the number of sources K when N increases. In other words, K may scale up with N, or it may stay 
constant regardless of N. 

From now on, we assume that the spectral norms of matrices (Bjv)at>i remain bounded when N — > 00, 
i.e. it exists b max > such that 

sup ||Bjv|| < b max < 00 (7) 

N>1 
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The eigenvalue distribution of R^ is characterized by the empirical distribution function of its eigen- 
values, namely 

F N (X) = ^caxd{Af ) :\ { k N) <\, k = l,...,M} 

where card denotes the cardinality of a set. For each A G R, the function F/v(A) gives the proportion of 
the eigenvalues of R^v which are lower than or equal to A. Its associated probability measure, denoted 
fiN, is given by d/tjv(A) = jj Y^k=i ^(^ ~~ ) anc * * s carr ied by R+. In order to characterize the 
asymptotic behavior of /tjv, it is in practice quite common to characterize the asymptotic behavior of its 
Stieltjes transform. If p, is a positive finite measure (i.e. /i(R) < oo), the Stieltjes transform of /x is the 
function ^ M of complex variable defined as 

•,W = (8) 

We recall the following well-known properties of the Stieltjes transform, which will be useful in the 
mathematical developments throughout the paper. 

Lemma 1. Let ^> „ be the Stieltjes transform of some positive finite measure \x (i.e. //(R) < oo), and let 
us denote as its support. Then, 

1) ^ is holomorphic on C\5 M . 

2) lim^ +00 -iy^^iy) = p(R) 

3) Vz G C\R, 



|Im(z)| 

where lm(z) denotes the imaginary part of z. Moreover, \/z G C\S^ it holds that 

4) G C+ z/ z G C+, where C+ is f/ie upper complex half plane. 

5) If is carried by R + , then z^^(z) G C + if z G C+. 

6) Conversely, if ^ is a function analytic in C+ satisfying 

• ^(z) a«c? 2^(2) belong to C+ if z £ C+ 

• sup y>1 < +°° 

f/ie/i, \I/ is Stieljes transform of a positive finite measure carried by R+. 

7) \/ip G C£°(R, R), (the set of compactly supported real-valued smooth functions defined on R), we 
have 

1 </?(A)d//(A) = — linilm 1 / ip(x)^fu(x + \y)dx \ 

JR 7T y+O [J R J 
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Having recalled these basic properties of the Stieltjes transform of a positive finite measure, let us 
now go back to the asymptotic characterization of the empirical measure p,N or, quite equivalently, its 
Stieltjes transform, which is defined for z £ C — IR + as 



m N lz 



dfi N (\) 



It is worth pointing out that mjv(^) can be expressed as the normalized trace of the resolvent matrix, 
which is a matrix-valued function defined as 

Q N (z) = (r n - zl M y l = (X N -£% - zluY 1 (11) 

namely rhpi{z) = igTr [Qn(z)]. Except (fT6l ). the following results can be more or less immediately 
derived from |[T0l (see also lfT6lO 



Theorem 1. There exists a deterministic probability distribution /xjy carried by R + such that fijy — fi^ 
converges in distribution almost surely towards when N — > oo. The measure p,N, referred to in what 
follows as the asymptotic eigenvalue distribution of matrix Rjv, is characterized by its Stieltjes transform 

171n(z) as 



f dfi N (X) 

m N {z) = / — (12) 

Jr + A-z 



which is a solution of the equation 



m N (z) = -^Tr 



-z(l + a 2 c N m N (z))l M + o- 2 (l - c N )l M + 



1 + o- 2 cnitin(z) 

for each z £ C — R+. Let TV(z) be the M X M matrix valued function defined on C — R+ by 



T N (z) 



-z(l + a cnitin(z))Im + cr (1 - cn)Im + 



(13) 



(14) 



1 + a 2 CNrriN(z) 

Then, Tn(z) is holomorphic on C — R+. Moreover, almost surely, 

lim (rh]y(z) — i71n(z)) = (15) 

AT— »oo 

for each z € C— K+. Finally, for each M— dimensional deterministic vectors utv, vjv such that sup N ||ujv|| < 
oo and sup N ||vjv|| < oo, ?Y /jo/Js that almost surely 

lim u$ (Q N (z) - T N (z)) v N = (16) 

N— s-oo 

/or eac/i z £ C — R+. 

Proof: Convergence of fiN — fJ-N towards as well as the fact that mjv(^) is a solution to (fT"3l is 
due to ifTOl . As for the result in ( fl5l ), it is a well known consequence of the convergence of fi^ — 
towards 0. ( fl6l) is proved in the Appendix [0 ■ 
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Theorem [TJ is pointing out that the entries of the resolvent Qn(z) are almost surely asymptotically 
close to the entries of the deterministic matrix function Tjv(-z) (this statement follows from (fT6l ) by 
selecting u^v and v^y as two columns of Im); and that its normalized trace, fhj<i{z) as defined in ( fTOl ), 
is almost surely asymptotically close to mjv(-z)> one of the solutions to the polynomial equation in (fT3T ). 
Furthermore, the random measure jj,^ is also almost surely equivalent (in distribution) to the deterministic 
measure fi^ in this asymptotic regime. 

We denote by Sn the support of this measure /ijv, which will play a very important role in the 
following. The characterization of Sn has been first presented in IfTTl , and is based on the study of 
the properties of function mjv(^) which, since it is a Stieltjes transform, is holomorphic on C\5/v and 
real-valued on R\Sn- In order to characterize Sn, we will also consider the function wj^(z), introduced 
in IfTTl , defined from uin^z) as follows 

wn{z) = z (l + a 2 CNmN(z)) 2 - cr 2 (l - c/v)(l + a 2 CNm N (z)). (17) 

It will be seen later on that the function wn(z) has very interesting properties that will be crucial for the 
derivations in this paper. In particular, we will show in the following that the support of hn, namely Sn, 
is in fact equal to the support of the imaginary part of wn(z) when z approaches the real axis. Thanks 
to this fact, we will be able to characterize the support Sn by studying the properties of wn(z) for z 
on the real axis. 

The next proposition provides some preliminary properties of mjv(z) and wn(z) that will become 
useful in the following sections. Most of these properties are established in IfTTl . We will denote by 
/n(w) the function on C — {71, . . . , 7m} defined by 



1 



f N (w) = —Tr (B N B N -wI 



which coincides with the Stieltjes transform of the eigenvalue distribution z^v(dA) = -h YlkLi ^(^ — Tfc) 
associated with the signal matrix B^B^. 

Proposition 1. The following properties hold: 

1) The condition cn < 1 implies that does not belong to Sjy. 

2) For each x € R, lim2 g c + ,z->-a; mN(z) exists, and will be denoted ni]\f(x). The function itin(z) thus 
defined is continuous on C+ U R, and continuously differentiable on C+ U IR — dSiy. Moreover, for 
each x G R, lim z6 c_. z -> x m^(z) exists, and is equal to (m^(x))*. The measure is absolutely 
continuous, its density is ilm(mjv(aO)> and the interior Int(<Sjv) of Sn is given by 

Int(5jv) = {x > : lm{m N (x)) > 0} (18) 
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3) For each x G R, lim zt =£ +)Z ^ x wn(z) exists, and is still denoted by wn(x). The function z — > wn(z) 
is continuous on C+ U IR, and is continuously differentiable on C+ U R — dSjy. Moreover, wn(x) = 
x (l + a 2 CNrriN(x)) 2 — a 2 (l — cn)(1 + o- 2 CNrriN(x)). Finally, lim z6 c _ >z ^ x wn{z) = wn{x)*. 

4) wn(x) does not belong to the set {7^ , . . . , 7^} if x £ R — 5/y. 

5) Im [wn{z)} > if Imz > 0. 

6) Re [l + C]\[a 2 mN(z)~\ > for each z G C. 

7) For any x £ R — 8Sn, the function m^{x) is solution of the equation in diil ) 

8) For any x 6 IR — 8Sn, the function wn(x) is a solution of the equation 

(Pn(w n (x)) = x (19) 

where 4>n(w) is defined by 

<f> N (w) = w(l- c N a 2 f N (w)) 2 + (1 - c N )a 2 (l - c N a 2 f N (w)) (20) 

Proof: Property Q] is not established in ifTTl . and is proved in Appendix lAl As for Property [2j the 
existence of the limit of mN{x + \y) is proved in IfTTl for x / because IfTTl did not assume that c^ < 1. 
However, Property Q] implies immediately that the limit exists if x = because 777.^(2;) is holomorphic 
in a neighborhood of the origin. The continuity and the differentiability of x — )■ 771,^(2;) is established in 
IfTTl on IR* and R*\dS N respectively, but it also holds on IR and [R\<9Sat by Property [JJ and the fact that 
171n(z) is holomorphic C\5at. Since mpf(z) is the Stieltjes transform of a positive measure, it is clear 
that 777tv(z*) coincides with m* N (z). This implies immediately that lim 2/< o i j / - > .o ttin(x + vy) = m* N {x). 
Finally, (fT8T ) is a direct consequence of the continuity of x — > m^{x). Property [3] follows directly from 
Property |2] Properties fj] and f5] are established in IfTTl . As for Property |6j it was initially proven in IfTTl 
for z G C*, but it can be shown easily that it holds for z = using Property [TJ as well as the proof of 
Lemma 2-1 of IfTTl . Finally, IfTTl established that mjv(^) is solution of ( fTBl if x G int(«Sjv)- This also 
holds if x G C\Sn because by Properties fj] and [6l the right hand side of (fT"3l is holomorphic on C\Sn- 
Since 7777V (z) is itself holomorphic on C\Sn, the equality in (fT"3l must hold not only on C\IR+ but also 
on C\Sn- Recalling that Sn is a closed set, all this implies that 777/v(x) is solution of equation (fT"3l for 
x G R\&Sjv. 

Let us finally establish Property [8] Thanks to Properties [6] and f7] and to (fT"3l) . we can write 

777 AT (x) 



1 + a 2 CNmN(x) 



f N (w N (x)) (21) 
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for each x G R\8Sn- This last equality can be rewritten as 

1 - a 2 c N f N (w N (x)) = — — — ^ (22) 

1 + a z CNmN{x) 

where the right hand side is well defined thanks to Property [6] Now, plugging (|22l into (fTTT ). we obtain 
that, for x G R\<95at, wn(x) is a solution of the equation 

4>N (w) = x (23) 
where function 4>n{w) is defined in d20l ). In other words, the function wn(x) satisfies (fT9l ) for each 

X G R\d<SjV ■ 

Proposition [TJ is establishing the fact that both m,N{z) and wn{z) are well defined when z approaches 
the real axis, and that mjv(:c) and wn(x) can be determined as one of the solutions to ([T3l and ([T9l ) 
respectively for any x G [R\<95at. In the next section we will establish some properties that characterize 
wn(x) out of the set of all the solutions of (fT9l ). and this will in turn help us in the characterization of 
the support Sn- 

IV. An alternative characterization of Sn 

In this section we will provide a characterization of the support Sn as a simpler alternative to the 
study provided in ifTTl . It must be pointed out that ifTTl assumed that the eigenvalue distribution of 
matrix B^B^ converges to a limit distribution ^(dA), and showed that hn converges towards a 
probability distribution [i^. Its Stieltjes transform is solution of (fT3l . but in which the discrete 
measure z^v(dA) = jj YljLi ^(^ — 7^) ^ s replaced by measure ^(dA), i.e. 



moo(z) 



A F 1 

-z(l + a 2 CNin N (z)) + a 2 (l - cn) + 5 ^oo(dA). 



1 + (T 2 CNmN(z)) _ 

In IfTTl . a detailed analysis of the support Soo of fj,^ was presented. The corresponding results provide 
of course a characterization of Sn by replacing the general probability distribution i^oo(dA) by the 



discrete measure i/j\r(dA) = Ylj=i ^(^ — 7fc )• However, we show in the following that it is possible 
to reformulate the results of IfTTl in a more explicit manner by taking into account immediately that 
~M T^f=i &{^~ 7fc ) i s a discrete measure. We hope that the following analysis, based on quite elementary 
technics, is easier to follow than the general approach of IfTTl . 

Our approach is based on the study of the function wn{z) that has been introduced in (fJTJ. We have 
established in Proposition [TJ that wn(x) is well defined in the real axis, and that it can be expressed as 
one of the roots of the polynomial equation in (fj9]). Let us now see how this function can help us in the 
characterization of the support Sn- 
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Proposition 2. The function wn(z) defined in (fT71) satisfies the following properties: 



1) Int (S N ) = {x G R+ : Im{u;jv(x)} > 0} 

2) > 0, for x G [R\cSAr. 

3) 1 - <j 2 c N f N {w N {x)) > Vx G R\5jv. 



Proof: See Appendix |B] 



Remark 3. #y taking derivatives with respect to x on both sides of the equation <Pn(wn(x)) = x, we 
see that w' n (x)4>' n (wn(x)) = 1 holds for x G R — 8Sn- Property [2] of the above proposition is thus 
equivalent to 



Property Q] in Proposition |2] is basically stating the fact that the interior of the support Sjv coincides 
the region of values of R + for which the imaginary part of wn(x) is strictly positive. Hence, it suffices 
to study the behavior of Im [wn(x)] in order to characterize the interior of the support Sjy. On the other 
hand, we know from Property [8] in Proposition Q] that, for any x G R\dSw, wn{x) is one of the solutions 
to the polynomial equation in (fl9l ). Proposition [2] is helping us to identify which one of the roots is in 
fact wn(x). More specifically, we will later show that: 



• If x G Int (5/v), then wn(x) will be the unique root of ( fT9l ) with positive imaginary paru, thanks 
to Property [T] 

• If x G R\Sn, then wn(x) will be the unique root of ( fT9l ) such that Properties |2] and [3] hold. 

In order to establish the fact that these properties completely determine the value of wn(x) out of the 
set of roots of the equation in ( fT9l ), we need to study the form of the function 4>n in d20l ) more closely. 
The analysis of the roots of the corresponding equation in ( fT9l ) will allow us to determine the intervals 
of R for which wn(x) is real-valued and the intervals in which it has a strictly positive imaginary part. 

A. Characterization of the function <p^(w) 



In the following, we assume that the K non-zero eigenvalues of the matrix B atB^, namely yy M _ K+l , ■ 
have multiplicity 1. Under this hypothesis, the equation in (fT9l ) is in fact equivalent to a polynomial 
equation of degree 2(K + 1). This can be readily seen by using the expression of /n(w) in (1201 ). so that 

'The existence and unicity of such root will be established in what follows. 



<j>' N (w N (x)) > if x G R\S N . 



(24) 
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we can express 4>n(w) as sums of quotients of polynomials in w, i.e. 

(M \ 2 

2 M-Kc N 2 c N s-^ 1 \ 
1 + a 1T7 TT > 
M W M 7m -w 
m=M—K+l / 

,/-, \ 2 ( i , 2 M ~ K °N 2 °N 

+ (1 - cn)ct 1 + cr — — a — > — 

\ M w M " 7 m 

\ m=M-K+l m 

nM 
{% 
rn=M—K+l 

a polynomial equation of degree 2(i<C + 1). If certain eigenvalues of ByvB^ are multiple, 4>n(w) = x 
will be a polynomial equation of degree 2{K + 1) where K represents the number of distinct non zero 
eigenvalues of BjvB^. The following results can thus be immediately adapted by replacing K by K. The 
assumption K = K allows to avoid the introduction of new notations representing the distinct eigenvalues 
of BatB^ in the forthcoming analysis. 

1) Zeros of 4>n{w): It is easily seen that the function <pN has exactly 2K + 2 different real zeros, 
which will be denoted as z^ N ^~ < Zq N ^ + < ... < z^~ < zj*p + . An elementary analysis of the 
function 4>n determines the position of these zeros, as well as the behavior of the function (j>N{w) in 
their neighborhood: 

• The lowest couple of zeros are located on the negative real axis, namely Zq N ^ ,Zq N ^ + G ]— oo,0[. 
Furthermore, the function 4>n is increasing at z^~ and decreasing at z^ + , namely (p' N (z^j > 
and cf)' N (zq^ + j < 0, where <fi' N denotes the derivative of 4>n. 

• The next couple of zeros are located between zero and the first positive eigenvalue of BjvB^, i.e. 



AN)- JN)+ 



®iiM-K+i ' anc ^ ^ turns out that the function 4>n is decreasing at z\ lw ' and 
increasing at z[ N ^ , namely 4>' N (z[ N ^J < and <p' N (z[ N ^ + j > 0. 
• Each one of the remaining couples of zeros is located between two positive eigenvalues of B jyB^ 



JV' 
(JV)- 



ie z {N) - z {N)+ e 



(AO (N) 
^M-K+k-V^M-K+k 



N 

, V k = 2, . . . , K, and the function 4>n is always 
decreasing at the first zero and increasing at the second, i.e. 4>' N (^z[ N ^ ^ < and 4>' N (zj^j > 0, 
V k = 2, . . . , K. 

In order to obtain these results, one only needs to factor 4>n{w) as the product of two terms, namely 

cj) N (w) = [l - c N a 2 f N (w)] [w (1 - c N a 2 f N (w)) + (1 - c N )a 2 ] (26) 

and therefore 4>n{w) = if and only if one of these two terms is zero. Out of the 2K + 2 zeros of the 
function 4>n(w), a total of K + 1 are the zeros of the first term in d26l ). More formally: 
• The second zero, namely Zq , is solution of the equation 1 — cj 2 cn /n{w) = 0. 
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• The zeros for k = 1, . . . , K are the solutions of the equation 1 — <j 2 cn /n(w) = 0. 

This allows us to differentiate between intervals of the real axis where 1 — ct 2 cjv In{w) > and 
intervals where 1 — a 2 CN Jn{w) < 0, namely 

• The function 1 — a 2 c^ fN{w) is positive on the intervals 



-OQ,Zq 



J N ) Z ( N )- 

IM-K+k-V z k 



}fc=l, 



7m ,+°° 



(27) 



This last fact is important, because we know from Property [3] of Proposition [2] that, when x does 
not belong to the support Sm, the solution of the equation 4>n(w) = x corresponding to w^{x) will 
be such that 1 — <j 2 cn /tv(u>at(x)) > 0, and therefore will be located inside of one of these intervals. 
In Figure Q] we give a typical representation of function ^tv(w) in a situation where K = 2 (we drop 
the dependence on iV in all quantities in the figure to simplify the representation). The function 4>n(w) 
presents horizontal asymptotes at w = and also at the values of the positive eigenvalues of B^B^, 
namely {7^-^+1 > ■ ■ ■ >7m^}- T ne re gi° n of the horizontal axis where 1 — g 2 cn In{w) > is shaded 
in grey. 




Figure 1. Typical representation of (f>M (w) as a function of w for K — 2 and Q — 2 (we drop the dependence on N for 
clarity). The shaded region in the horizontal axis represents the set of points for which 1 — (j 2 C]v/jv(to) > 0. The shaded region 
in the vertical axis represents Sn- 
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2) Local extrema and monotonicity intervals of 4>n(w): Next, we investigate the local extrema of 
the function 4>n- The following proposition summarizes the most interesting properties of the positive 
local extrema. 



Proposition 3. 1) The function (j)^ admits 2Q positive local extrema counting multiplicities (with 



1 < Q < K + 1) whose preimages, denoted Wy < < w[ N ^ 
belong to the set {w G IR : 1 — a 2 CNfN(w) > 0} 



< w 



(N)- 



. . . < W 



(N)- 



< W 



(N)- 
Q 



2) If we denote by x^ 



3) Each eigenvalue 7j o/BjvB^ belongs to one and only one of the intervals 
q = l...Q. 

4) The function 0tv is increasing on the intervals 
. Moreover, 



and x 



(JV)H 



A N) ~ 



0<x\ iy > <xY' <x 



... < X 



(N)- 



these positive extrema, then 



< X 



(28) 



w 



(JV)- 



(N)- 



, W 



(N)- 
9+1 



} 

J o=l,Q-l 



and 



W n ,+oo 



-OO, W 



4>N 



W 



(JV)- 
9+1 



'(N)+ JN)- 



IV. 



{N)~ 
Q 



+oo 



-oo,x\ 

for each q = 1, 

(A^)+ 

, +oo 



,Q — 1, and 



Proof: Except for the inequalities in (1281 ). which are proved in Appendix the statements of 
Proposition [3] follow directly from an elementary analysis of the function 4>n. ■ 
We see from Proposition [3] that the local extrema always appear in groups of two, and the actual 
number of extremum couples (Q) will generally depend on a 2 , cn and on the positive eigenvalues of 
the matrix BatB^. For example, in the situation represented in Figure [TJ the number of positive local 
extrema was equal to four, which implies that Q = 2. In Figures [2] and [3] we depict other equivalent 
examples of 4>n, for which we had Q = 1 and Q = 3 respectively. 



B. Characterization of wn(x) out of the roots of 4>]\r(w) = x 

We know from Proposition[TJthat wn{%) for real valued x will be a solution of the equation 4>n(w) = x. 
In this section, we will characterize which one of these roots is actually wn(x). First of all, observe 
that, since the equation (/)n(w) = x is equivalent to a polynomial equation of degree 2{K + 1), the 
number of solutions (counting multiplicities) will always be equal to 2{K + 1). Out of these solutions, 
we can graphically find the real- valued ones by exploring the crossings between the graph of 4>n(w) 
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Figure 2. Typical representation of <pM (to) as a function of w for K — 2 and Q = 1 (we drop the dependence on N for 
clarity). The shaded region in the horizontal axis represents the set of points for which 1 — a 2 cat /at > 0. The shaded region 
in the vertical axis represents Sn- 



and a horizontal line at x. This is further illustrated in Figure [4] By the properties of the function 
4>N (w) presented in Section HV-AI we can clearly differentiate between two different situations: 



if x i uJLi 



X 



(AO- JN)+' 



' X k 



, it is easily shown that the equation 4>n{w) = x presents exactly 
2(K + 1) different real-valued solutions (cf. upper horizontal line in Figure |4]). Since the original 
equation has degree 2{K + 1), there are no complex-valued solutions. In particular, wn(x) will be 
real-valued. 

• If x € UfcLi x k : ' i n wnat follows, it will be shown that the equation 4>n{w) = x has 

exactly 2K different real-valued solutions (cf. lower horizontal line in Figure |U). This implies that 
there is a couple of complex conjugated solutions to the equation 4>n(w) = x. 

Let us now see how we can completely characterize wn(x) in these two different situations: 



I) Case x £ R\\J 



k=l 



: From (l24l) and Property [3] of Proposition |2l we know that 
wn(x) is a root of the equation 4>n{w) = x such that 4>' n (wn(x)) > and that 1 — ct 2 cjv /jv(^jv(^)) > 0. 
We now prove that this completely characterizes wjy(x) out of the set of all roots of 4>n{w) = x, in 
the sense that there is only one root of 4>n(w) = x that has these two properties. We first consider 
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\ w ; r 




Figure 3. Typical representation of <j>M (w) as a function of w for K — 2 and Q = 3 (we drop the dependence on N for 
clarity). The shaded region in the horizontal axis represents the set of points for which 1 — (j 2 Cjv/jv(u>) > 0. The shaded region 
in the vertical axis represents Sn- 



the case x < . By Property @] of Proposition 4>n is an increasing one to one correspondence 



from 



-oo, w 



onto 



-oo, X 



(TV)- 



onto 



-oo, w 



(N)- 



. Its inverse (f)^ 1 is thus a well defined increasing function from 
We claim that wn(x) coincides with (f>~^ (x). Indeed, observe that 
since <p]^(x) < w[ , we automatically have (j)' N ((f>]^(x)) > and that 1 — a 2 CNfN{4'N 1 ( x )) > ^ n 
the other hand, the behavior of 0jy established in Propositions |2] and [3] implies that the other real-valued 
solutions of <f>N(w) = x do not satisfy either 1 - a 2 CNfN{w) > or (f>']y{w) > (see further Figures 
[T]to [3). Therefore, wn(x) can be expressed as <j)Z- (x), and is the only root of 4>n{w) = x such that 
1 — <J 2 CNfN(w) > and 4> N (w) > 0. 



The above analysis can be extended if x belongs to 



x > x 
for k 



(N)- 



S N )+ J N ) 



' X k+1 



for k 



1, . . . , Q — 1 or if 



Indeed, observe first that 4>N is a bijection between 



1, 



x 



(N)+ JN) 



' X k+1 



Q — 1 and between 
for k = 1, . . . , Q — 1 and on 



w o , +oo 



and 



x 



(N)+ 



w 



(N)+ (N) 



, +oo 



Hence, t 



and 



x 



is well defined on 



+oo 



. Thanks to the form of the function 0jv> we 



see that (^^(x) is the only root that verifies 1 — cr 2 CNfN(w) > and 4>' N (w) > (see further Figures 
Q] to [3]), and this implies that wn(x) = ^^(x). Since wn (x) is continuous on R, we also get that 



w N (xl. 



(N)- 



(N)- 



as well as wn(x^ + ) 



w 



(JV)H 



for k = 1, . . . , Q. 
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Figure 4. One can find the real-valued solutions to 4>m (w) = x by examining the crossings of the graph (f>M (w) with a 
horizontal line at x. In this particular example, where K = 2, we see that 0jv (w) = x presents 2(K + 1) = 6 real-valued 
solutions, whereas <j>N (w) = x' has only 4 real-valued solutions (plus a couple of complex conjugated ones). 



2) Case x e Ufc=i 



W- JN)+ 



: In this situation, we establish that the equation 4>n{w) = x has 
exactly 2K real-valued solutions, plus a couple of complex conjugated ones, and that wn(x) is equal 
to the complex-value root with strictly positive imaginary part. We can reason from the behavior of 4>n 
that the polynomial equation 4>n(w) = x has at least 2K real- valued solutions located in the intervals 



(AT) 

^M-K+l-V Z l 



and 



+ (AO 
Z l i^M-K+l 



for I = 1, . . . , K. We however note that none of them can satisfy 



both 4> N {w) > and l — a 2 CNfN(w) > 0. Therefore, wn(x) cannot coincide with one of these solutions. 
Assume that the two remaining solutions of the equation are real. u>n{x) of course coincides with one 
of these two solutions. The properties of function 4>n as well as (|28T ) imply the existence of two extrema 
of 4>n , denoted by x* < x\ such that x G Moreover, by d28l ), the two extra solutions must 



belong to an interval 



+ (AO 
Z l i^M-K+l 



for I = 1,...,Q — 1. Consequently, these two solutions satisfy 
1 — a 2 CN Jn(w) < 0, and cannot coincide with wn(x), which leads us to contradiction. Therefore, the 
two remaing solutions are complex conjugate, and wn(x) coincides the solution with strictly imaginary 
part. 
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C. Characterization of the support Sn 

As the interior of Sn coincides with {x £ R+, lm(wN(x)) > 0} (see Property Q] of Proposition |2]), we 
have shown the following Theorem. 



Theorem 2. The support Sn is given by 



U 

k=l 



X 



W- JN)+ 



The above analysis shows that x\ " ' < x\ A "'^ < x y 2 ' ' < 



- x Q-l — X Q 



(29) 



< x^q^ + coincide 



with the set of all positive extrema of <pN- Theorem [2] thus establishes a very simple method to determine 
the support S^. First, one needs to determine all the local extrema of 4>n (w), namely the solutions to 
the polynomial equation 4>' N (w) = 0. The solutions will be , w[ N ^ + , . . . , WqP ,Wq^ + \ with 

possible repetitions if one of these roots has multiplicity two, plus K additional ones (it is easily seen that 
4>N has exactly K negative local minima). By evaluating the function 4>n at these points, and selecting 
those for which 4>N is positive, we are determining the values ,x[ N ^ + , . . . ,Xq^ , aig^ j that 

characterize the support in d29l ). Observe that the support Sn is a disjoint reunion of compact intervals, 



which will be referred to as clusters. Each of these clusters 



interval of the type 

,X N ) 



w 



(*0- J N )+ 

i w q 



X 



W- JN)+ 



5 ^q 



,q = l...< 



}, in the sense that x q N ^ 



will be associated to an 



4>N Wq 



(N) 



and x 



(N)- 



On the other hand, we can also clearly see that a specific eigenvalue 7^ ,& = !,..., M, 



(N)- (N)+ 



This motivates the following 



always belongs to one, and only one of the intervals 
definition. 

Definition 1. We say that the eigenvalue 7! , k = M, of the matrix B^rB^ is associated with 



the cluster 



SN)- JN)+ 



j ^q 



III 



(AT)- (JV)+ 



It'n 



Observe that this is not a one-to-one correspondence, in the sense that multiple consecutive eigenvalues 
of BtvB^ may be associated with the same cluster. For instance, in Figure |2] the three eigenvalues (0, 
7m-i an d 7m^) 316 associated with the same eigenvalue cluster, while in Figure [3] each eigenvalue is 
associated with its own different cluster. 

The first cluster [x± , x^ ] plays a special role because it is always associated with the eigenvalue 
of matrix B^B^. As seen below, the main results of this paper will be valid under the assumption that 
the strictly positive eigenvalues of ByvB^ are not associated to the cluster [x[ N ^ ,x[ N ^ + ]. Intuitively, 
this means that the eigenvalues corresponding to the noise subspace are separated from the eigenvalues 
of the signal subspace. Both Figure [J and Figure [3] satisfy this property, but not Figure |2l 
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More rigorously, we assume from now on that the following hypotheses hold. 
(As 1): 3Nq G IN such that ViV G IN, N > Nn, the non zero eigenvalues \ of 

I K J k=M-K+l,...,M 

B^rB^ are not associated to the first cluster [x± ,x^ + ]. 

(As 2): > 0,tf,t2 G R independent of N such that 

t7 < inf ix\ N) ~ \ < sup ix\ N)+ \ <t+ <t~ < inf (x^°~) ViV > iV . (30) 

1 N>N I 1 J N~>Nq I J W>iV I J 

These two assumptions imply that for each iV > iVo, the eigenvalue of B^B^ belongs to the interval 

wj 7 ^ - , w[ N ^ + and thus to ] WN(t^), WN(tf) [ because WN(t±) < w[ N ' and wx(tf) > u^ - * - . 
Similarly, the non zero eigenvalues {tj^x+z} of B^B^ satisfy "f^-K+l > w Jv(4)' 



V. Convergence and localization of the sample eigenvalues 

The previous results are related to the properties of the limit deterministic distribution p^. The almost 
sure convergence of pn~ fJ-N towards does not mean by itself that the eigenvalues of Ryv belong almost 
surely to Sn, or to an interval containing Sn- As one may imagine, it is important to be able to locate 
the eigenvalues {X^)k=i....,M of matrix R^ with respect to Sn for N large enough. Bai and Silverstein 
established in lfT2l . |fT3l powerful related results in the context of correlated zero-mean, possibly non 
Gaussian, random matrices. In the following, we establish similar results for the Information plus Noise 
model. However, the mathematical approach we use in the present paper has no connection with the 
techniques used in lfT2l . |fT3l also valid in the non Gaussian case. Since S is assumed Gaussian, we 
rather adapt to the Information plus Noise model the ideas developed in [14] in the context of Gaussian 
Wigner matrices. We prove in the following two theorems which are believed to be of independent 
interest. 

Theorem 3. Assume that there exists a positive quantity e > 0, two real values a,b G R, and an integer 
Nq such that 

]a-e,b + e[nS N = ViV G N,JV > N (31) 

where Sn denotes the support of pn- Then, with probability one, no eigenvalue o/Rat appears in [a,b] 
for all N large enough. 
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Theorem 4. If Assumptions\7]and\2\hold, then, for all N large enough, with probability one, 

\[ N , . . . , X^-k e ] *i ' *i" [ (32) 

(33) 



A M-K+1 > z 2 



Although Assumptions [U and |2] depend on the deterministic distributions ^n, Theorem |4] shows that 
almost surely, the smallest M — K eigenvalues of R^r are always separated from the others for all N 
large enough. 



A. Proof of Theorem \3\ 

We first state the following proposition, the proof of which is demanding, and is detailed in Appendix 
[Ej The result will play a fundamental role in the proof of Theorem [3] 

Proposition 4. \/z e C\R+, we have for N large enough, 



E 



^Tt[T n (z)} + ^ X n(z) 



with x ^ analytic in C — K+ and satisfies 

1x^)1 ^(M + c^PGim^r 1 ) 



(34) 



for each z £ C+ where C is a constant, k is an integer independent of N and P is a polynomial with 
positive coefficients independent of N. 

We now follow lfl8l and lfl4l and prove the Lemma: 

Lemma 2. Let <f> be a compactly supported real-valued smooth function defined on R, i.e. <p G C£°(IR, IR). 
77iJ 



E 



1 

M 



Tr [0 {V N V N H )] 



0(A)d^(A) = o( w ; 

iSjV 



(35) 



Proof: We first note that, by Property [7] in Lemma [Q we can write 



as well as 



^(A)d^(A) 





i(s)E 







— lim Im 

7T j40 



— lim Im < I 0{x) 



— Tr [Qjv(x + iy) 



d.r 



— Tc[Tjv(x + i|/)] 



d.r 



By applying the function cj> to a Hermitian matrix, we implicitly represent the action of (f> on the corresponding eigenvalues. 
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Therefore, using Proposition 01 we can express the right hand side of (l35l) as 



j <s at 



{x)xn{x + \y)dx 



(36) 



Since the function xn(z) satisfies the inequality (l34l) . the Appendix of |[T9l implies that 



lim sup 

2/4-0 



v{x)xn{x + iy)dx 



<C< +oo 



where C is a constant independent of N. Hence, (1361 ) readily implies (I33T ). 

In order to establish Theorem |3l we consider a function ?/> G C%°(R, R) satisfying < tp < 1 and 

1 for A G [a, 6] 

for A G R - ]a - e, 6 + e[ 
Condition (f3TT > implies that f s ^ ^(A)d///v(A) = if N is large enough. Therefore, (l35l) implies that 



z/>(A) 



E 



1 

M 



Tr (S^)] 



O 



N 2 



We now establish that 



Var 



(37) 



In order to prove (l37l ). we use the Nash-Poincare inequality |[20l . ETI . |[22l . lfT4l which implies that 



Var 



9 



aw. 



M 



Tr [V> (SatS^ )] 



+ 







^Trf^E^)] 



(38) 



where Wy denotes the (i,j)th entry of matrix W defined in Now, applying e.g. |[T8l Lemma 4.6] 
we can readily see that 

8 r ^Tr[^(S^)]l=^[^^(^)] 



d 



M 



^Tr (E*E#)] j = ^ [V (E*E£) Ejv] 



(39) 
(40) 



where ■0' denotes the derivative of if). Consequently, the sum on the right hand side of (I38T ) can be written 

as 



'•j 







1 

M 



Tr (EatS^ )] 



+ 



S 



aw 



L 



^Tr (E„E#)] 



M 2 



Tr 



This yields 



Var 



M 



(E^E^)] 2 E^E^ 
(EjvE^fEjvE^ 



(41) 
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for some constant C independent of N. Next, consider the function h{\), denned as h(X) = A [^'(A)] , 
which clearly belongs to C£°(R, R). Lemma |2] implies that 



E 



i-Tr 

M 



MA)d^(A) + 0(^ 



But it is clear from ( 1311) that f s h(X)dp^(\) = if N is large enough. Therefore, (|4~TT) gives ( f37T >. 

We are now in position to complete the proof of Theorem [3] as in lfl4l . Applying the classical Markov 
inequality together with the above results, we can write (for N large enough) 

2" 



P (i-TY (E.E*)] > < iV^E 



1 



AT 8 / 3 



E 



-TV (E*E#)] 



M 

2 



+ Var 



-Tr (S^E^)] 



O 



AT4/3 



(42) 



Then, by Borel-Cantelli lemma, for N large enough, we have with probability one, 



AT4/3 



By the very definition of ip, the number of eigenvalues of R^r = EtvE^ in [a, b] is upper-bounded by 
Tr [^(EatE^)] and is therefore a 0(N~s) with probability one. Since this number has to be an integer, 
we deduce that for N large enough, there is no eigenvalue in [a, b]. This completes the proof of Theorem 

El 



B. Proof of Theorem @ 

The approach we use to establish Theorem [4] differs from the method of lfl4l which is inspired by 
lTT3l . The first part our proof is similar to the proof of Theorem |3l and thus we will omit certain details. 
For the second part, we will need a certain result that we summarize in the following proposition: 

Proposition 5. Consider the curve C defined by the complex valued function wn (x) in di7D on the 
complex plane as x moves from to tf, concatenated with the function w^(x) as x moves back from 
t~± to t±, namely 

C = {w N (x) : x £ [ti,4]} U {w* N (x) : x G [t^,tf]} . (43) 

This is a closed curve that encloses the points of] , [ (see further Figure\5\. Let i/)(z) be a function 
holomorphic in a neighborhood of C. Then, the contour integral J c _ if>(\) dA is well defined by 



(p V(A)dA = 2iIm / ^(wn{x))w' n (x) dx 
Jc- J [t!,tt] 



(44) 
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where w' N (z) denotes the derivative of wn(z) and where the symbol C means that C is oriented 
clockwise. 



Finally, let £ G R a point that does not belong to [wpf(t 1 



. w 



(N)- 



U w 



w N(tt)]- Then, 



Ind.K) = - 



if £ G 



(JV)- 



,W 1 



if £ < ) or £ > w N (tf), 



Proof: According to the discussion in Section IIV-B1 if x G 



and increases from w^(t 1 ) to I x\ 



w 



(N)- 



. For x G 



^1 ' • c i 



S*0- J N )+ 



i 



, then wn{x) is real-valued, 
, the point wn(x) belongs 



to C+. Finally, if x G 



,wn(x) is again real- valued, and increases from wn ix\ 



w 



(N)- 



,w 1 



J 



W 



(N)- 



to u>N(tf). The contour C is therefore well defined and encloses the points of 

Let us now prove (l44l ). Observe that the function x — > wn{%) is not exactly a piecewise continuously 
differentiable function on [t^" , t ^ ] because |u>^r(x)| increases without bound when 
To see that wn(x) can indeed be used as a valid parametrization of C, we need to see that the integral in 
(1441 is well defined. It is thus necessary to study the behavior of w' N around the points , j. 

The following lemma is an immediate consequence of the analysis of the behavior of the density of 
measure fi^ near a point of OSn provided in ifTTl (see Appendix iDl for a proof). 



Lemma 3. There exists neighborhoods V ( x^ 

C 



(JV)- 



and V ( x 



SN) 



(N)- 



of x\ and x\ such that 



and 



\w' N {x + iy)\ < 



\w' N (x + iy)\ < 



x — X 



(N)- 



for y > 0, x + iy G V(x^ ), and x ^ x\ 



(45) 



C 



for y > 0, x + iy G V(x[ N ^ + ) and x ^ x\ 



x — x 



(46) 



In particular, Lemma [3] implies that Jj t _ t +, \iP(wn(x))\\w' n (x) \ dx < +oo so that the right hand side of 
(1441 is well defined. The reader may check that it is possible to use the usual results related to integrals over 
piecewise continuously differentiable contours. In particular, as lm(wN(x)) > if x G , a;^ 



the index of a point £ G R which does not belong to wn (t 1 ) , 



1 is £ G 



U 



w 



(N)- 



WN(tt) is equal to 



and to if either £ < WN(t^) or w > w^itf). ■ 
Proposition [5] is basically pointing out that the function wn{x) defines a valid parametrization of a 
contour that will not intersect with any eigenvalue of B^vB^. Furthermore, Assumptions Q] and [2] imply 
that 

Indc(O) = 1 (47) 
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and 

lnd c (jM-K+i) = (48) 

for I = 1,...,K. This means that the contour will only enclose the zero eigenvalue, and none of the 
positive eigenvalues of BjvB^, which will be of crucial importance in the following development. Figure 
|5j gives a schematic representation of the form of the contour C. 




Figure 5. Representation of the contour C on the complex plane. 



Having introduced the result in Proposition [5j we are now in the position of establishing the proof of 
Theorem |U Let ip G C£°(R, R) such that < tp < 1 and 



V>(A) 



1 VAG[t^,t+] 
VA € R - [ti - e, t£ + e] 
with e chosen in such a way that tf + e < ij. Since ^ £ C£°(R, R), we can use Lemma [2] to get 



E 



1 

M 



Tr (SjvE^)] 



^(A)d/ijv(A) + 



1 

]V2 



Assumptions Q] and |2] imply that 

^(A)d/xjv(A) = /ijv 



W- (N)+ 
1 > x l 
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for N large enough. This leads to 



E 



M[*r.*i"])+° 



i 

iV2 



As established in (1371) . we also have 

Var 



^Tr^(^Sf)] 



1 

N 1 



because supp^^nS/v = for N large enough. Therefore, using again the proof of theorem [3] (inequality 
(|42~1)). we get that 



i^Tr [0 (S,^)] - ^ ([t r ,t+]) = O (_Lj) 



a.s. 



(49) 



Let us now find a closed form expression for fi^{[ti ,t^]). Noting that hn is absolutely continuous with 
density -Im(mN(x)), we can write 

([*r>*i1) = - lm 



7T 



By expressing the Stieltjes transform as m^{x) 
be written as 

m ([*r>*f]) = - Im 



mAr(x)dx 

^)) (see further ( g2j) ! MJV ([*-*+]) can 



1— a 2 c N f N (w N (x)) 
1+1 fN(w N (x)) 



-dx 



! t - I - a 2 c N f N (w N (x)) 

In order to express /^jv ([£T>*i]) ^ n terms °f an integral over the contour C, we can use the relation 
w' n (x)(I)' n (wn(x)) = 1 for each x G IR — <95/v (see further (fl9l)). Now, using Proposition [51 we see that 

/jv(A)^(A) 



M[*r,*i1) = 

/iv(A) 



1 



Im 



7T 



fN(w N (x))(j)' N (w N (x)) , 

— ■ 2 7 7 / \ \ w N {x)ax 

t-,t+] 1 - a z c N f N {w N (x)) 



1 



27ri J c - 1 - a 2 c N f N (X) 



dX 



. (1 - c N <r 2 f N {\)) 2 - 2c N a 2 Xf' N (X) (1 - c N cr 2 f N (\)) - c N a 4 {\ - c N )f' N (X)) 



2vri 



1 - (J 2 c N f N (X) 



dX 



(50) 



The integrand of the right hand side of (1501 is a meromorphic function. The contour integral can be thus 
evaluated using the residue theorem. The poles of the integrand are the eigenvalues of BjyB^ as well as 
the solutions of the equation 1 — o 2 cn //y(A) = 0. This equation has K + 1 real-valued solutions that we 
have denoted Zq N ^ + , and ^zj N ^ j (see further Figures Q] to [3j). Assumptions Q] and |2] imply that 

only the poles {0} and |^o^ + 1 of the integrand are in fact enclosed by C. Using the residue theorem, 
and after some straightforward calculations, we obtain a closed form for the above integral, namely 



M-K {N) J_ 
M 1 + M 



M 

£ 

k=M-K+l 



a 



(N) 
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with 



Ql ~M-K\ M ^ JM} z {N)+ 

\ i=M-K+l ll / z 

a W = L _ K \J_ o*(l-c N ) 

a * y 1 W 7 w + ,w + _ 7 w (52) 



Therefore, we can write 




M 



(N)+ (N) 
k=M-K+l z ~ 7fc . 



M 

but, using the fact that 1 — a 2 CN /jv(^o ) = 0> we obtain that /ijv ([^n^i"]) = M m < ■ Inserting this 
into (|49l), we get 

Tr^(^S^)]-(M-^) = o(^) 

with probability 1. Moreover, thanks to theorem [3j no eigenvalue of Xl^X^ appears in [t± — e, tj"] U 
[tj", almost surely for iV large enough. Therefore, almost surely for N large enough, Tr [^(^atX!^)] 

coincides with the number of eigenvalues of Sat Xjy contained in the interval 1 i i" , i i" [• This number is 
thus equal to M — K. These eigenvalues are moreover the M — K smallest ones: otherwise the smallest 
eigenvalue of SjyS^ would belong to [0,£f], a contradiction by Theorem [3] Finally, Theorem [3] again 
implies that ^m-K+i > t%. This completes the proof of Theorem [4] 

VI. Consistent estimation of the localization function 

We now present a consistent estimator t]n = ^n^-n^n of the subspace method localization function. 
Here, bjy represents a M-dimensional deterministic vector, and we assume that sup N ||bjv|| < oo. The 
new consistent estimator presented in this section can be seen as an extension of the work in Q, which 
implicitely assumes that the useful signals are Gaussian random i.i.d. sequences. In order to simplify the 
notation, we drop the dependence on N from all the sample eigenvalues and sample eigenvectors. 

Theorem 5. Under Assumptions \J\ and \2\ we have with probability one, 
where ff^f w is defined by 

M 

VN W = J2^ h N^h N (55) 



(54) 



fe=i 



September 30, 2011 



DRAFT 



29 



Here, the coefficients |£fc| 

it' 

6 



1 + 



Af 



Afc + A; 



are given by 

+ a 2 (l - c N ) 



(A fc - A,) 2 



M 

£ 

;=M-if+l 



Afc — A; 



LO, 



(56) 



ant/ < > Z?y 
I J k=M-K+l,...,M 



2 M-K 

£ <7 Cat A fc + A 2 2/i \ 



1 



1 



i=1 (Afc - A;) 2 ^ VAfc — A; Xk — LOi, 

and where {uji} 1=1 M represent the solutions (arranged in increasing order) of the equation 

-11 



1 + ^Tr 
M 



M i 



0. 



(57) 



(58) 



We remark that the consistent estimator is a linear combination of the terms (b^^ej^b/v), _ 1 M - 
In contrast to the traditional estimator r]t ra d = Sfcii^ b^efce^bTv, it contains contributions of both the 
noise subspace and the signal subspace. We also note that the assumptions Q] and |2] and Theorem |4] are 
intuitively important because the various sums on the right hand side of d56l ) and (I57T ) remain bounded: 



in (l56l ) and (l57l ). the terms 



Afc -A; 



are greater than t 2 —tf, and it will be shown that a similar property 



holds for the terms 



Afc - 



Remark 4. It is worth pointing out that whenever the number of samples is forced to be much larger 
than the observation dimension (N » M or equivalently cn — > 0), the proposed estimator converges 
to the classical sample eigenvector estimate. This can be readily seen by taking the limit as cn —> in 
the coefficients of ( |56| ) and ( 1571 ) and noticing that uii — > Xi when cn — > 0. Hence, as cn — > we have 
£ k -> lfork= 1,...,M-K, andik -> 0fork = M-K + l,.. .,M, implying that ffff w - fj% ad -> 0. 
ITu's shows that the proposed estimator is in fact a generalization of the classical sample eigenvector 
estimate. 



The remaining of this section is devoted to presenting the main points of the proof of Theorem [5] The 
starting point consists in remarking that Assumptions Q] and [2] imply that 



^jf b# (B N B% - AI M ) 1 bjydA 
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where C is the closed path defined by (1431) . This leads to 
rj N = — (BatB^ - w N (x)I M ) b N w' N (x)dx+ 

- ^~ (B N B% - ^(x)Im)" 1 b^ (^(x))* dx = 

~Im fjf 1 b$ (BtvB^ - ^(x)I M ) _1 b Jv ^(x)dxj . (59) 

Let g N (x + iy) = b^ (BjvB^ - w N (x + iy)I M ) _1 b^w^x + iy). The function y g N {x + iy) is 
continuous on R + for each x G R\<35at thanks to Proposition Q] Lemma[3]and the dominated convergence 
theorem imply that 

r] N = lim -Im ( / b$ (BjvB^ - w N (x + vy)l M ) 1 h N w' N (x + iy)dx\ (60) 
y\a TT \J tI J 

If 1 f y 1 f y 

7T^ f 9N{z)dz-— g N (q+ih)dh + — g N (t^-ih)dh 

27n JffKZ 2n J-v 27r J-v 



lim 

ViO L^ITl J dn - 4TX J_y Z7T J_y 

where d1Z~ is the boundary (clockwise oriented) of the rectangle 1Z y defined for y > by 



(61) 



TZ y = {u + iv :u e ,tf],v £ [-y,y]} . (62) 

Notice that the last two integrands vanish as y J, (since the function u i— )• gN^i + if) is continuous on 
[-y,y]), and thus 

777V = lim — & g N {z)dz. 
vio 2vri J dn - 

Moreover, since gN(z) is holomorphic in C^x^ ,x[ N ^ + ], the value of the contour integral does not 
depend on y > 0, and therefore the limit can be dropped, namely 

VN = tt-t f gN(z)dz. 



2vri 



Using the equality (1 + a 2 cm^{z))(^N'B 1 ^ — wn(z)Im) 1 = Tjv(z), which follows easily from the 
definition in (fl4l ). we can write 



5JV (>) = b#TV(z)b 



1 + a 2 cmi\r(z) 

Now, the key point of the proof is based on the observation that gN{z) can be estimated consistently 
from the elements of matrix Rjy. We recall that rhj\f(z) is defined by 

1 1 A/ 1 
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and we define wn(z) as the function obtained by replacing function mjv(^) with riiN{z) in the definition 
of wn{z), i.e. 

W N (z) = Z (l + <J 2 CNlh N (z)) - <7 2 (1 - C/y) (l + a 2 c N ?h N (z)) (64) 
We define the corresponding random asymptotic equivalent of qn{z) by 



g N (z) = b N Q N (z)h 



w' N (z) 



N 1 + a 2 CNlTlN(z) 

Observe from the definition of mjv and of Qtv that the function is meromorphic with poles at 
\\,...,\m and at Cj\,...,&m, the M real-valued solutions to the polynomial equation (of degree M) 
1 + <j 2 CNmN{x) = 0. In the following, it is important to locate the (&i)i=i r ..,M- 

Lemma 4. For N large enough, with probability one 

Ai, • • • , Xm-K-,^1, ■ ■ ■ ,&m-k £]*i"> hi (65) 
Xm-k+i, Xm,&m-K+i, •••,wjf are greater than t% (66) 

Theorem Q] implies that almost surely, qn{z) — 9n{z) — > on 972- 3/ \{t 1 ", i^}. In order to be able to 
use the dominated convergence theorem, we first state the following inequalities proven in Appendix [Hj 
there exists Nq £ IN such that 

sup sup |<7jv(z)| < +oo (67) 

N>N z&dTLy 



and 



sup sup |<7jv(2)| < +oo (68) 

N>N zedTly 



almost surely. The dominated convergence theorem thus implies that 

■> a.s. 



1 

2vri 



9n(z) - gN(z)dz 



We now establish that the integral 

m? w = 

2vri 

is equal to ff}f w defined by (|55l ). This can be shown using residue Theorem. 

Lemma [4] implies that for N large enough 

M-K 

-new = ^ [ind^ (x k ^J Res (g N , A fc ) + lnd dn - (u k ) Res {g N ,^k) 

k=l 

where Res(<77v, A) denotes the residue of function g^ at point A. 
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In order to evaluate these residues, we first remark that 



k=l 



A fc - z 



ijN (z) can thus be written as 

9N (z) = ^2 h Nh&k h N 



M 



fe=l 



a fc (» + /3fc(z) + 7fc(z) 



where we have defined 



1 + a 2 CNrfiN(z) 
A fc - z 

2<7 2 CArZ77l Ar (z) 



Afc - z 

m'^z) 



Afc - z) (1 + a 2 c N m N (z)) 



7fc(z) = -a Cjv(l - cat) 



and consequently with probability one for N large enough 

M M-K 

-new = ^2 h%e k e k h N ^ [Res (a k , X m j + Res (j3 k , A m ) + Res ( %, A m ) + Res (% 
fc=l m=l 
Classical residue calculus gives 



Res (afc, Ar; 



o C N 1 



W A fc -A 

1 + ^ 2 cjv4E 



2„ J_ v^Af 1 



k ^ m 
k = m 



Res /3fc,Ar; 



Res (%, \ r 



2a 2 c N At 



M 



1 (Afc— Am) 



2(t 2 c jv v^M Afc 



k 7^ m 



k = m 



Res (7fc,w m ) = -a 2 



1 - CAT 
Afc — tDrn 



-i i a- 2 Cjv V"^M 1 

j^fc A *~ A " 



k ^ m 
k = m 



Next, we define £fc as 

M— _ff 
Ifc = - X] Res 



«fc, A m J + Res [j3 k , \ m ) + Res ( %, A m ) + Res (%, u>. 



m=l 
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We obtain, for k = 1, . . . , M - K 



1 



M 



M 

E — 

i=M-K+l Ak Al 



+ 



2a 2 c N 
M 



M 

E 

--M-K+1 Afc - A 



+ M 



1 ~ c N 

CN 



M-K 



M-K 



M 



+ a 2 (l-c N ) 
and for k = M - K + 1, . . . , M 



\ 



E — i - - E — — + E — - 

j = i Aj — Afc i=1 — Afc i=1 Aj — Afc 

j^fc i^k i^k 



n M-K 

<r c N 



M 



Er 1 



_ 2 M-K t 

2a z c N v-^ A fc 



E 



u Ai — Aft M 



a\l-c N ) J2 



U>i — \i 



i=l 



Aj — Afc I ( &i — Afc 



To retrieve the final form of given in the statement of the theorem, we notice that 



M 



1 1 

1 + <7 CJ\T— — / 

and use the following lemma proved in Appendix U 



(76) 



(77) 



(78) 



(79) 



Lemma 5. The following identity holds for any k = 1 . . . M 

1^ 1 2^ 1 1^ 1 

M hi^-*k M h^-^ M hi&i-^ 

i^k i^k ij^k 

This establishes that fj new = fj new and completes the proof of Theorem [5] 

VII. Numerical results 

In this section, we compare the results provided by the traditional subspace estimate, the new estimate 
d55l ) (referred to in the figure as the "conditional estimator"), and the improved estimate of Q derived 
under the assumption that the source signals are i.i.d. sequences (referred to as the "unconditional 
estimator"). 

We consider a uniform linear array of antennas the elements of which are located at half the wavelenght. 
The steering vector a(6>) is thus given by 



a(0) 



1 



'M 



l,e 



in sin(6>) 



3 i(M-l)7rsin(6») 



(80) 



In the following numerical experiments, source signals are realizations of mutually independent unit 
variance AR(1) sequences with correlation coefficient 0.9. In order to evaluate the performance of the 



September 30, 2011 



DRAFT 



34 



various estimators, we use Monte Carlo simulations. The additive noise varies from trials to trials, but, 
for fixed M and N, matrix S remains unchanged. Finally, unless otherwise stated, the cluster associated 
to the eigenvalue of matrix AS is assumed to be separated from the clusters corresponding to its non 
zero eigenvalues, i.e. for each a 2 , M and N, it holds that 

o < 4 N)+ < 4 N ] ~ < iSUi (si) 

We finally mention that the estimate of is supposed to be unconsistent in the context of the following 
experiments because the source signals are not i.i.d. sequences. However, we will see that the performance 
of the conditional and the unconditional estimates are quite close, a property which will need further 
work (see Remark [5]). 

Experiment 1: We first consider two closely spaced sources, i.e. 9\ = 16° and 62 = 18°. The number of 
antennas is M = 20 and the number of snapshots is N = 40. The separation condition (f8TT > is verified if 
the SNR is larger than 10 dB. In order to evaluate the performance of the estimates of the localization 
function, for each improved estimator (conditional and unconditional), we plot versus 9 in figure [6] the 
ratio of the MSE of the traditional estimator of a(#)^na(#) over the MSE of the improved estimator. The 
SNR is equal to 16 dB. Figure [6] shows that the 2 improved estimates have nearly the same performance, 
and that they outperform significantly the traditional approach around the 2 angles. We however notice 
that the 3 estimates have nearly the same performance if 9 is far away from 6\ = 16° and 62 = 18°. 
In order to evaluate more precisely the improvements provided by the conditional and the unconditional 
estimators around 9\ and 6> 2 , we plot vs SNR in figure [7] the mean of the MSEs of the estimates of 
a(0i) H IIa(0i) and a((9 2 ) H na(6> 2 ). 

In figure [U we plot for each method the mean of the MSE of the two estimated angles versus the SNR. 
The estimates of 9\ and 92 are defined as the arguments of the two deepest local minima of the estimated 
localization function. The mean of the two Cramer-Rao bounds is also represented. The performance of 
the 2 improved estimates are again quite similar, and they provide an improvement of 4 dB w.r.t the 
traditional estimator in the range 15dB-25dB. 

We now plot the probability of outlier, i.e. the probability that one of the two estimated angles is 
separated from the true one by more than half of the separation between the two true sources. In figure 
|9l we compare the outlier probability of the three approaches versus the SNR of the three estimators. For 
a target probability of error of 0.5, the 2 improved estimators provide a gain of 8 dB over the traditional 
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10 



10 



Traditional Estimator 
-a — Conditional estimator 
-* — Unconditional estimator 
CRB 



15 



20 25 
SNR 



30 



35 



Figure 8. Mean of the MSE of the angles estimates versus SNR 



estimate. 




10 



— Traditional Estimator 
-a — Conditional estimator 
-* — Unconditional estimator . 



15 



20 25 
SNR 



30 



35 



Figure 9. Outlier Probability vs the SNR 
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We finally evaluate the influence of M and ./V on the performance. N varies from 20 to 200 while the 
ratio c/v is kept constant to 0.5, and SNR = 15 dB. In figure [10] we have plotted the mean of the MSEs 
on the estimates of a(0j) IIa(0j) for i = 1, 2. The separation condition ( f8TT > occurs for N > 32. Figure 
[TOl illustrates clearly the unconsistency of that the traditional estimate. 




Figure 10. MSE for the estimators of the localization function vs N 

Experiment 2: We now assume that the number of sources K is of the same order of magnitude that 
M and N, i.e. K = 10, M = 20, N = 40. The ten angles (0i) i=1 ,„ M io are equal to 0, = -40° + (i-l)10° 
for i = 1, . . . , 10. The separation condition holds if SNR is greater than 15 dB. We again plot versus 
in figure [TT] the ratio of the MSE of the traditional estimator of the localization function over the MSE 
of its conditional and unconditional estimators. SNR is equal to 16 dB. Figure [TT] shows again that the 
performance improvement of the conditional and unconditional estimates is optimum around the angles 

(#j)i=l,...,10- 

Figure [T2l represents the mean of the MSEs of the various estimates of a.(6i) H Ha.(9i) for i = 1, . . . , 10 
w.r.t. the SNR, and confirms the superiority of the 2 improved estimates when the separation condition 
(I8T1) . We note that 

Remark 5. All the previous plots clearly show that the conditional estimator outperforms the traditional 
one, while its difference with the unconditional one is negligible. Tins is a quite surprising fact. To explain 
this, we recall that the unconditional estimator has been derived in /[7]/ under the assumption that matrix 
Sm is a Gaussian matrix with unit variance i.i.d. entries. The unconditional estimator of fi7§ is based on 
the observation that if Sjv is an Ltd. Gaussian matrix, then the entries of (Rat — zl)~ 1 have the same 
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Figure 11. Ratio (in dB) of the MSE of the traditional estimate of the localization function over the MSE of its improved 
estimates versus 9 




Figure 12. Mean of the MSE of the estimates of a(0i) H IIa(0i) for i = 1, . . . , 10 versus SNR 

behaviour than the entries of matrix T?Niid( z ) defined by the following equation 
m Ntiid (z) = --^Tr T N)iid (z) 

T N ,ud{z) = [(AA H + <7 2 l) (1 c/v c N zm N ^ id {z)) - zl] 1 

One can verify that the entries ofTN(z) defined by 1114$ . which depend on Sjy, have the same asymptotic 
behaviour than the entries ofTN,ud{ z ) when Sn is a realization of an i.i.d. matrix. In this case, the 
conditional and unconditional estimators have of course the same behaviour. If however S n is not an 
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i.i.d. matrix, then the entries of (Rat — zT)" 1 do not behave like the entries of TN,ud( z ) so that the 
unconditional estimator should become unconsistent. The previous simulation results tend to indicate that 
it is not the case. The explanation of this phenomenon is a topic for further researchs. 

VIII. Conclusions 

This paper has considered the use of subspace estimation algorithms in situations where the number of 
available samples and the observation dimension are comparable in magnitude. We have considered the 
information plus noise signal model, according to which the received signals are deterministic unknowns 
whose empirical spatial correlation matrix is low-rank. We have derived an estimator of the noise subspace 
of the spatial correlation matrix that is consistent, not only when the number of samples tends to infinity 
for a fixed observation dimension, but also when these two quantities increase to infinity at the same 
rate. This guarantees that the estimator will present a good performance even when these two quantities 
are comparable in magnitude. In order to establish the consistency of the estimator, we have proven new 
results concerning the almost sure location of the eigenvalues of the sample covariance matrix of an 
Information plus Noise Gaussian model. 



Appendix A 
Proof of Property[T]of Proposition Q] 

In order to establish that does not belong to the support Sn, we show that it exists e > for which 
Hn([0, x]) = for each x g]0, e[). In order to show this, we will make us of the function h(m, z) defined 
as 



1 



h(m,z) = —TV 



-z(l + a CNm)I M + o (1 — cn)Im + 



1 + a 2 CNm 



(82) 



Observe that the equation m = h(m,0) is equivalent to 

1 



m = — Tr 

M 



a (1- c N )I M + 



1 + a 2 C]ym 

Now, the condition cn < 1 implies that the function m — > is decreasing on R + . Therefore, the 

equation m = h(m, 0) has a unique strictly positive solution denoted m*. Next, we will check that 

dh 



1 



dm 



> 0. 



(83) 



(m,,0) 



Indeed, observe that 



dh 
dm 



a 2 c N 



1 



(m.,0) 



1 + <7 2 CAr7Tl* M 



Tr 



BjyB^ 

1 + a 2 CNm* 



CJ 2 (1 - C N )l M + 



BjyB^ 

1 + a 2 CNm* 
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so that 



dh 
dm 



^ o 2 cn 1 T 
< 7. rr lr 



(m.,0) 



1 + (J 2 c^m^ M 



2/i \ t BatB^ 

cr (1 - c/v)Ia/ + 



1 + a 2 c N m^ 



1 + <r 2 c N m if _ 

as required. Hence, the implicit function theorem implies that there exists an open disk centered at 
zero with radius rj > 0, i.e. D(0,rj), and a unique function m(z), holomorphic on D(0,rj), satisfying 
m(0) = to* and such that 

m(z) = h(m(z),z) (84) 

for \z\ < rj. Evaluating the successive derivatives of function z — > h(m(z),z) at the origin, one can 

check that for each I > 0, m^(0) is real- valued. Since to* > 0, there exists a positive quantity e, 

< e < ?7 such that m(x) is real-valued and m(x) > if x e] — e, e[. On the other hand, it can be 

readily checked that if x < 0, the equation m = h(m, x) has a unique strictly positive solution. Now, for 

x < 0, ?tin(x) is strictly positive, and satisfies this equation. Therefore, it holds that 771^(2) = m(x) for 

— e < x < 0. Since the two functions uin and m are holomorphic on D(0, e)\ {[0, e[} and coincide on a 

set of values with an accumulation point, they must coincide on the whole domain of analicity, namely 

D(0, e)\ {[0, e[}. We recall that for < x < e, /i/v([0, x]) can be expressed as 

1 f x 
(jLn([0, x]) = — lim / Im(mjv(s + iy))ds 
7r y->o,y>oj 



Therefore, 



1 r — 

Miv( 0, xl) = — lim / Im(m(s + iy))ds 
IT y^0,y>0 J 



As m is holomorphic on Z)(0,e), the dominated convergence theorem implies that 

— lim / Im(m(s + iy))ds = — Im(m(s))ds = 
7T y~>Q,y>0 J 7T 7 

because m(s) G R if s G [0,.x]. This establishes that jUjv([0, x]) = 0. 

Appendix B 
Proof of Proposition [2] 

In order to prove Property [T] we establish that lm(wN(x)) > if and only if Im(mjv(x)) > 0. 
Assume that Im(mjv(x)) > 0, i.e. that x G Int(<Sjv), which in particular implies that x > 0, and consider 
z = x + iy with y > 0. Equation ( fT3T ) can be written in terms of wn(z) as 



1 + CN& 2 mN(z) 



f N (w N (z)). (85) 
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Taking the imaginary part from both sides yields the identity 

- Im(w N (z))±-Tr [(B N B% - ^(z)^)" 1 ^!^ - ^(^Im)" 1 ] 



\l + a 2 c N m N (z)\ 2 v JVV "M 
or equivalently, 

Im(m N {z)) = lm{w N {z))\l + a 2 c N m N (z)\ 2 -^Tr [(B N B% - w N (z))-\B N B% - ^(z))" 1 ] 

(86) 

= lm(w N (z))^Tv [T N (z)T%{z)] (87) 
It is shown in ifTTl (see Eq. (2.6)) that 

^Tr [T N (z)T«(z)] < -L < I 

which implies 

Im(mivO)) < Im(wjy(g)) 9 , , . (88) 

If y — >■ 0, we get that 

< Im(mj\r(a;)) < Im(wAr(x)) 



<7 2 C/v|x| 



which implies that lm(wN(x)) > 0. Conversely, assume that 1ui(wn{x)) > 0. Then, m^{x) cannot be 
real-valued, otherwise, wn{x) = x(l + a 2 CNmN(x)) 2 — cr 2 (l — cjv)(1 + a 2 CNmj^(x)) would be also 
real-valued. 



Next, we prove Property|2] Since x — > mjv(x) is differentiable on K— <3Sat, x — )• wn(x) is differentiable 
on the same subset. By Property [4] of Proposition [TJ wn(x) does not belong to the spectrum of matrix 
BnBn if x G R\«Sjv. Therefore, the function x — > /at(iujv(:e)) is differentiable for x G R\5at. Since 
d85l ) holds on i £ IR\«S;v> we can differentiate it with respect to x on x G IR\5jv- This gives 

Wjv(aO/Ar(wiv(aO) = /-, , 2 
(1 + cn<t mN{x)y 

for x G K\5jv. Now, observe that m' N (x) > on IR\5jv because mj^(z) is the Stieltjes transform of a 
probability measure carried by Sn- On the other hand, the function f' N is of course strictly positive on 
R. This in turn shows that w' N (x) > on x G IR\«Sjv. 

To establish the last property, we use (fT3l) at point x G R\<Sjv, and get that 

1 - c N a 2 f N (w(x)) = — \ — . (89) 

The conclusion follows from the inequality 1 + cno- 2 i71n(x) > if x G IR\5jv (see Proposition [Q). 
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Appendix C 
Proof of (1281) in Proposition [3] 

We consider w\,W2 G {^1^ j w[ N ^ + , . . . ,V]q^ ,Wq^ + \, and denote by <pi and (p2 the quantities 
^n{w\) and <Pn(w2) respectively. We define Ai n = 1 — (J 2 cn fN{w n ) so that we can write n = u^/i 2 + 
ct 2 (1 — CN)h n , n £ {1,2}. Our objective is to show that the quantity ((fo — </>i) / (f#2 — wi) is always 
positive. Note that, by definition, w\ and W2 are inflexion points of 4>n(w) such that /ii > and hi > 0. 
Using direct substraction of the expressions of 4>i and 4>2 we can write 

. ri2 - hi 



W2 — Wi 

Consider now the following inequality 



/, . , ■, (w 2 h 2 -w 1 h 1 ) 2/1 ^2 
+ h 2 ) h a (1 - ctvJ- 



W2 — W\ 



(AT) 

7fc 



M 



M 

/ (AO 



^2 



1 M 
- M ^ 



(AT) 
7fc 



W2 — Wl 



/ll/l2 



fc=i (7fc 



(AT) 



10 1 



(AT) 

f J_y- ±1 

M fci(7r-- 2 ) 2 



(90) 



which can be readily obtained by noting that 



1 M 
M ^ 



/ 



fe=i 



(AT) 

7fc 



1/2 



(7 



(TV) 



w i) (7fc -^2) 



> 0. 



Using the definition of hi and /12 we can readily write 

w 2 h 2 - w\hi _ ^ a 2 CN 
U)2 — w\ 

and hence the inequality in (l90l) is giving us 

(j} 2 - 4>\ 



(AT) 

7fc 



W2 — W\ 



> {hi + h 2 ) 



1 



a 2 c N 



(/at(wi) + /at(w 2 ) + wif' N (wi) + W2f' N {w 2 )) 



+ 



AiiAio+^d 



ho - /ii 



CAT 



(91) 

^2 — ^1 

where f' N (w) denotes the derivative of Jn(w). Using again the definition of Aii and hi, we can rewrite 
the last term of the previous expression as 



ho — h 



a 2 c N 



W2 — W\ 



M 



(W 2 ~ Wif 



<> ( W \ 2 / (AT) 

fc =i(7fc "^i) 2 (7fc 



W 2 ) 



By inserting this last equality into d9TT ) and replacing /at(^i) with a 2 (1 — /ii), we obtain the expression 

M , , 2 



W2 — W\ 



Q- 4 Cjy(l - c at) 1 



(W2 - Wx)" 



T X] , (AT) 

JW fc =i (7fc - ^i) 2 (7fc - w 2 y 



+ 



/l 2 + /l 2 



+ 



C7 4 CAr(l - C/v) 



[/aK^i) + fifM] 



a 



hi + Ai2 



2 (wi/jV(wi) +^2/^(^2)) 



• (92) 
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Now, both w\ and w 2 are preimages of local extrema of 4>n, so that for n = 1, 2, we have 4>' N (w n ) 
hn — 2a 2 w n f N (w n )h n — <r 4 (l — CN)f N (w n ) = 0. Thus, we can write 



h\ + h 2 



a' 



[wihif' N (wi) + w 2 h 2 f' N (w 2 )] + 



a A c N {l - c N ) 



and by inserting the last equality into d92l ), we obtain 



a 4 c N (l - c N ) 1 



M 



(w 2 - Wl)' 



W 2 — W\ 



^S(7f ) --i) 2 (7f ) -- 2 ) 2 



+ y(^i - h 2 )(wif' N (w 1 ) - w 2 f' N (w 2 )). 

(93) 



M 



Using again the fact that (f>' N (w n ) = 0, we can write w n f' N (w n ) = J^t 
becomes 

M 



<T 2 (1-C N ) f' N {w n ) 



and thus d93 



^2 — 01 > cr 4 c N (l - c N ) 1 ^ 



(w 2 - WiY 



w 2 — W\ 



Clearly, we have 

M 



M ^ 



2 M 

<?\l-c N ) 
4 

(w 2 - Wx) 2 



, (hi ~ h 2 ? | 



a 4 (l - c N ) 



-CN 



(94) 



fc^i (7* - ^i) 2 (7fc VJ - w rf fe= i (7fc ' - ^ijl7fc ' - W2) 

and thus by multiplying the previous equality with hih 2 and adding /^/^(^i) + h 2 f' N {w 2 ), we can also 
write 

1 M 

h 2 2 f' N (w 1 )+h 2 1 f N (w 2 ) + -J2 



2 

M 



M 



1 



(AT) 



h\h 2 {w 2 — wi) 



M ^Ail N) -^) 2 {ir>-^) 2 

-hth 2 [ f > N ( Wl )+f> N ( W2 )] = ±f2 

The left hand side of the previous equality appears in d94l as a common factor on the last two terms of 
the right hand side of that equation. Hence, plugging it into (|94l ), we obtain 

M , , 2 



- + 



M 



hn 



/M 



w 2 — W\ 



H Q- 4 Cjy(l - cat) 1 



M 



(W2 - Wl)' 



s(7f ) -^) 2 (7r ; -^) 2 



+ 



f (fei - h 2 ) 2 + cr 4 CAf(l - cat) 1 ^ 



/? 



4/ii /i 2 



M 



11,12 1V± k =i \% ' ~ Wl % ' ~w 2/ 
Finally, noting that all the terms of the above equation are non-negative, we have established (l28l) . 
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Appendix D 
Proof of Lemma [3] 

The proof of this Lemma is a direct consequence of ifTTl Section 4]. Next, we provide some details 
on how to obtain d45l ); the same procedure can be applied in order to obtain (|46l ). As in IfTTl , we define 
in this section function btq{z) by b^^z) = 1 + a 2 CNm^{z) for z £ C, and denote by the quantity 
bfq{xi) (note that we drop the dependence on N in x~[). Since x± belongs to 8Sn, both misr(xi) and 
b^ are real-valued. Proposition [JJ thus implies that z — > biq{z) is continuous at the point x±. Similarly, 
wn{x~[) = w± is real- valued so that the function z — > wn(z) is also continuous at x±. 

Since f' N {w^) > 0, there exists a neighborhood V(wf ) of on which /jy is biholomorphic. For 
z G C + U R, it follows from (fJJJ that we can write 

/»<»»«) = , - T w ? , = 4- f 1 - rn) < 95 > 

1 + a^c N m N {z) g^cn \ on\z) J 
Since wn is continuous at x± and since u>at(z) G C+ if z G C+ (see Property f5] of Proposition [TJ), there 
exists a neighborhood V(xj) of xj~ such that 

w N (V(x^) n C+) c V(wf) n C+. 

Therefore, applying the holomorphic inverse of /jv, denoted as , to both sides of (|95T ) we get, for 

any z G V(xf ) n C + , 

Using the fact that wn(z) = zb 2 N (z) — a 2 (l — CAr)6jv(z) and solving with respect to z, we get that 

z = Z N (b N (z)) z G V(xi) n C + (96) 
where Zn is the function defined in an appropriate neighborhood of b± by 

Next, we recall the following result from IfTTl . 

Lemma 6. There exists a neighborhood V(6j~) of b^ and a function \fjv, biholomorphic from V(b^) 
onto a neighborhood of the origin V(0) such that V6 G V(fcf) 

Zjv(6)-xr = *^(6). 
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Since the function is continuous at the point x^[, and since bjy{z) G C+ if z G C+ (which follows 
from the definition of 6/v), there exist two smaller neighborhoods V'(x^) C V(x^) and V'(b^) C V(frf) 
of x^ and 6^ respectively, such that 

b N (z) g v'(&7/) n c+ Vz g v' (x^) n c+ 

Therefore, using (l96l ). we can write 

(tfjv (M^))) 2 = z-a?r 
\/z G V'(a^) n C+. Let us now choose, \/z G V'(x^) n C+, 



^at (6jv(z)) = \J z ~ x \ 

where -^/(-J represents any determination of the complex square root that is holomorphic^ on C + and 
such that \/T = 1 (the following reasoning applies verbatim to the square root determination for which 
\/T = —1). We denote by the holomorphic inverse function of defined on V(0). We have 

b N (z) = ^ (Jz-xij Vz G V'(x^) n C+ 

Taking derivatives with respect to z at both sides of the previous equality, we obtain 



2^z-x\ V 



Now, since ^ ^ is holomorphic on V(0) by Lemma [6j the function [^jy 1 ]' wm ^ e bounded on the same 
neighborhood of and thus we will have 



z — Xi 



for some constant C independent of z. Therefore, for z = x + iy G V'(x l ) n C+, we can write 

C 

\x — x^ + iy 



w' N (x + iy)| = \b N (z) 2 + 2zb' N {z) - a 2 (l - c N )b' N (z)\ < = (97) 



The inequality 



C C 



\x — x 1 + iy| y \x — x-l | 
for x ^ x^ completes the proof of d45l ) for y > 0. (|45T ) for y = follows from the observation that 

w' N (x) = lim^o w' N {x + iy). 

3 This property must hold for all possible choices of ^jv because, by definition, >l/jv is holomorphic on V(b^) and &jv(z) 6 C+ 
if z £ C+. Since bN^z) is holomorphic on V'(a;J~) D C+, ^jv (6jv(2)) must be holomorphic on the same set. 
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Appendix E 
Proof of Proposition @] 

In this section, we drop as much as possible the subscript N for an easier reading. In the following, 
Pi(|z|) and P2( iim^i ) represent generic positive coefficients polynomials of the variables \z\ and ^r^ji 
whose mean feature is to be independent of N. The values of Pi and P2 can change from one line to 
another. 



We rely extensively on the results of the Appendix II of E3l related to the properties of matrix 
(B + D 1 / 2 WD 1 / 2 )(B + D 1 / 2 WD 1 / 2 )^ where D and D are deterministic diagonal matrix. We thus use 
ll23l in the case where D = uIm and D = cjIjv which corresponds to the context of the present paper. 
In order to help the reader, we use the same notations as in |[23l all along this section. More precisely, 
we define 



1 



5(z) = acm(z) 
S(z) = 6(z)-a 



a(z) = E — TrQ(z) 



a(z) = a(z) — a 



1 



(98) 
(99) 
(100) 
(101) 



We remark that a(z) is the Stieltjes transform of measure caco where lo is the probability measure carried 
by \R + defined by 

u(B) = E(fi(B)) (102) 

for each Borel set B. We recall that jl represents the empirical eigenvalue distribution of Rjy = ^n^n- 
Finally, it is easily seen that 5 is the Stieltjes transform of measure acfi + c(l — c)6q (So represents the 
Dirac distribution at 0), and that ot(z), which can be expressed by 



a(z) = E 



^^TrQ(z) 



where Q(^) is defined by 



Q(z) 



fE^S-zir 1 



(103) 



(104) 



coincides with the Stieltjes transform of measure acw + cr(l — c)5q. 
Matrix T(z) defined by (fT4b can be written as 

BB^ 



T(z) 



-z(l + a5(z))I M + 



l + aS(z) 
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and 5(z) is equal to 

We also define matrix T(z) by 

T(z) 



5(z) = a-TrT(z) 



-z(l + a5(z))l N + 



B g B 

l + a~5{z) 



and remark, after simple calculations, that 



S(z) = er-TrT(z) 



We finally denote by R(z) and R(z) the matrices defined by 



R(z) 
R(z) 



-0(1 + ctq(z))Ia/ + 
-2(1 + aa(z))lN + 



BB 



1 + era (2 J 
B^B 



(105) 

(106) 

(107) 

(108) 
(109) 



1 + aa{z) 

Using Property [6]of LemmaQ} it is easily checked that functions (—z(l + a5(z)))~ 1 , (—z(l + cr8(z)) 
(—2(1 + cra(z)))" 1 , (— z(l + cra^)))" 1 are Stieltjes transforms of probability measures carried by R + . 
Proposition 5.1 of |[T6l thus implies that matrix valued functions T(z), T(z), R(z), R(z) are holomorphic 
in C — R + , coincide with the Stieltjes transforms of positive matrix valued measures carried by R + , the 
mass of which are equal to I, and their spectral norms are bounded by [CT^ji on C+ (see 1 161 f° r more 
details). 

We finally recall that matrices Q(z) and Q(z) satisfy ||Q|| < (im(z)) -1 and ||Q|| < (Im(z))^ 1 for 
z E C+ (see e.g. ED, HI, O, (El). 



In order to establish Proposition HI we have first to study the term 



E(-TrQ(z) 



N 



-TrR(z) 



A. Study of E (^Tr Q(z)) - ^TrR(; 
Let f{z) and A(z) defined by 



f{z) 



-a 



" x _ / B g E[Q(z)]B 

z(l + cra(z)) L N \ l + cra(z) 



(110) 
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and 



A(z) := A 1 (z) + A 2 (z) + A 3 (z) 
a 



A 2 (z) := 



A,(z) :-- 



1 + cra(z) 

r 2 



E 



Q(z)VV H -Tr(Q(z)-E[Q(z)]) 



1 + aa(z) 



;Q(z)-E[Q(z)])^-TrS^Q(z)B 



N 



a 



a 



As it will become apparent below, the entries of matrix A(z) converge towards 0. 
It is proved in ll23l that for each z G R*_, the following equality holds true 



f BB \ 

I M + A(z) = E [Q(z)] ( -z{\ + or(z))l M + 1 + (ja{z) ) 



(111) 
(112) 

(113) 

(114) 



(115) 



+ <ra{z) _ 

As the lefthandside and the righthandside of (11151 ) are analytic on C — R + , Eq. dll5l ) holds not only on 
\R~, but on C — IR+. It is shown in [23] that a{z) — f(z) converges towards for each z € C — R+ when 
N — > +oo. The general expression of a{z) — f(z) given in E31 is complicated. However, the simplicity 
of the model considered in this paper (matrices D and D in ll23l are reduced to oT) allows to derive the 
following Lemma. 



Lemma 7. For each z G C — IR+, it holds that 



z(a(z)-f(z)) = -a-TrA(z) 



Proof: Multiplying (II 151 ) from both sides by a and taking the trace, we obtain 



From the definition of f{z) (equation dl 101 )), we also have 

1 /B"E[Q(z)]B 



a — Tr . 

N \ l + aa(z) 

The two above equalities imply that 



zf{z){l + aa{z)) + a 



Using (1101b . we get that 



a(z) - t(z) = ^ - --Tr A(z) 



a{z) - f(z) = -—Tr A(z) 



(116) 



(117) 



(118) 



(119) 



(120) 



and Jim 
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Writing the righthandside of (II 151) as 

E(Q(^))R(^)- 1 + za(a(z) - f (z))E(Q(*)) 
and using (111 6b . we obtain immediately that 

E(Q(z)) - R(z) = A(z)K(z) +a 2 ± [TV A(z)] E(Q(z))R(z) 

and that 

lTrQ(z) - ^TrR(z) = ^Tr (E [Q(*)] R(z)) ^Tr A(z) + -^Tr A(z)R(z) 
The above expression of E [^TrQ(z)] — -^TrR(z) allows to prove the following Proposition. 



(121) 



(122) 



Proposition 6. \/z e C+, we have 

1 



E 



N 



TrQ(z) 



N 



TrR(z) 



^ M Pi(k))P 2 (|Im(z)|- 1 ; 



(123) 



Proof: We first prove the following preliminary result. 



Lemma 8. Consider M X M matrices Utv and M X N matrices Tj' N satisfying sup N ||Ujv|| < 
oo,sup7v ||Ujv|| < oo. Then, we have Vz € C+ 



Var 



Var 
1 

N 



N 



TrQ(z)U 



Tr£^Q(z)U' 



<c^iiut Pi(N) p 2( _l_) 



(124) 
(125) 



where the polynomials Pi and P2 and constant C are independent of M, N and U, U . 



Proof: As the proofs of the two statements are similar, we just prove the first statement of the 
Lemma. We first remark that 



d[Q(z)] pq 
<9W*. 



-Qpi (S H Q) i 

-Q i9 (QS) 



VJ 



(126) 
(127) 
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The Nash-Poincare inequality gives 



Var 



< 



< 



N 



1 ^ d[Q(z)]pg . 
N 2^ 



p.q 



dWi. 



<-W 



+ E 



1 d[Q(z)] pq 
N ^ 



c>W*. 

p,q l J 



'■j 



E | [X H Q(z)UQ(z)] ri + E | [Q(z)UQ(z)E 



1 y 



AT3 

We use the resolvent identity 



C W E E [(S H Q(z)^Q(z)^Q(z)UQ(z)5])^ 

i 

< C^E [Tr (Q(z)UQ(z)Q(z) H U H Q(z) // SI] H )] + 
C-^E [Tr (q^U^QW^QWUQ^SS^)] 



+ 



Q(z)SS H = SS ff Q(z) = I + zQ(z) 



Therefore, 



Var 



-TrQ(z)U 



1 



< Cj^ E |Tr (Q(z)UQ(z)Q(z)"u" (I + z*Q(z) H ))\ + 



E |Tr (Q(^U H Q(z) H Q(z)U (I + zQ(z))) 
<C\\Vf 



+ 



N 2 V|Im(^)| 4 |Im(z 



1 



<c||u||^(H + i) 

which establishes the first statement of Lemma [8] 



1 1 

+ 



|Im(»| 4 \1m(z) 



(128) 

(129) 

(130) 

(131) 

(132) 
(133) 

(134) 

(135) 
(136) 
(137) 

(138) 



We now complete the proof of Proposition [6] For this, we use the inequalities ||Q(z)|| < ij^^i an d 
\R(z)\\ < for z G C - R. This leads to 

1 



£tt(E[Q(z)]R(*))£ltA(*; 



< c 



|Im(z) 



±Tr A 1 (z) + Itt A 2 (z) + ±Tr A 3 (z] 



We establish that 



^Tr(A,(z)) 



< ^ f Pi(k|)P 2 (|Im(^)r 1 ) 



(139) 



(140) 



for i = 1, 2, 3. In order to evaluate -^Tr(Aj(2:)) for i = 1,2, 3, we first remark that 



< 



\z(l + aa(z))\ ' \lm(z)\ 
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because — z ( 1+ l a ( z ^ is the Stieltjes transform of a probability measure. Therefore, we have 

1 \z\ 

< 



The resolvent identity ( 11341 ) implies that 

-E 
-E 



|1 + aa(z)\ \Im(z 



^TrQ(z)^Tr(Q(z)-EQ(z)) 



a 



^ (Tr (Q(z) - EQ(z))) £ (Tr (Q(z) - EQ(z))) 



(141) 

(142) 
(143) 



1 + aa{z) 

(11411 ) and the first statement of Lemma [8] give immediately (11401) for i = 1. Similarly, -^Tr(A2(^)) can 
be written as 



^Tr(A 2 (z)) 



o 



■ E 



-^TrQ(z) - E(lTrQ(z)] (^TrS^Q(z)B - E(^TrV H Q(z)B] 



1 + aa(z 

Using again (1141b . the Schwartz inequality, Lemma[8j and the identity (xy) 1 / 2 < (^i^) for x > 0, y > 0, 
we get (11401 ) for i = 2. (11401 ) for i = 3 is obtained similarly. This and (11391 ) imply that 



^Tr (E[Q(*)]R(*))£ltA(z) 



^ ^Pi(kl)p 2 (|im(z)r 1 ; 



Using the same approach and the identity ||R(z)|| < (|Im(^)|) we obtain easily that 



N 



Tr A(z)R(z) 



<^Pi(kl)p 2 (|im(z)r 1 ; 



(|122l) thus implies Proposition [6l 



Remark 6. A is also possible to establish that \/z 6 C+, we have 



^TrQ(z) 



— Tr R(z' 



< ^P 1 (|z|)P 2 (|Im(z)|- 1 ; 



(144) 



because it is shown in H23\l that a relation similar to U15\) holds for E(Q(z)). Following the derivation of 
H121i , we obtain an expression ofE -^TrQ(z) — i'IrR(z) similar to M22\) which allows to establish 

dm. 



B. Study of E (^TrQ(z)) - ^TrT(z) 

In order to complete the proof of Proposition 01 we show in this paragraph that 



a 



( ^TrQ(z)) - IttTW 



\a(z)-6(z)\<j^P 1 (\z\)F 2 (\lm( i 



(145) 
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for each z £ C+. For this, we denote by e(z) and e{z) the terms defined by 



1 



e(z) = a(z) - a— Tr(R(z)) = a E— Tr(Q(z)) - — Tr(R(») 



N 
1 



f 



1 



e(z) = a(z) - a-Tr(H(z)) = a E-Tr(Q(z)) - -Tr(R(z)) 



N 



Proposition [6] and Remark [6] imply that 



"iV 
1 



N 
1 



"iV 



AT 



|6WI<^Pi(NI)P 2 (|Im(z)r 1 ) 
I^I^^Pi^DPsdMz)!- 1 ) 



(146) 
(147) 

(148) 
(149) 



for each z £ C+. In order to study a(z)—5(z), we express a(z) as a(z) = cr-^ r Tr(R(z))+e(z). Therefore, 
a(»-<5(» = a±Tr(R(z)-T(z)) + e(z). We have similarly d(z)-J(z) = a^Tr(R(z) -T(z)) + e(z). 
We remark that R(z) - T(z) can be written as K{z) (T _1 (z) - R^ 1 ^)) T <»> and that R (^) ~ T(z) is 
equal K(z) (t -1 0) -R _1 (z)) f(z). Using the expression of R(z) _1 , T(z) _1 , R(z) -1 and T(z)" 1 , 



we obtain that 



where 



a(z) — 5(z) 
a(z) — 5{z) 



D (*) 



a(z) — 
a(z) — 5(z) 

Uq{z) ZVq(z) 

zv (z) u (z) 





( e(z) \ 


H 




\ '<*) ) 



with uq , uq , vq , -Dq defined by 



_ 1 a 2 R(z)BB fl T(z) 

U °^~iV (l + <ra(z))(l + <7<5(z)) 
=1 ^R(,)B^Bf( f) 

N {I + aa(z)){\ + a5{z)) 

v (z) =lWR(z)T(z) 
Sd(*) =±Tra 2 R(z)T(z) 



(150) 
(151) 

(152) 
(153) 

(154) 
(155) 



Using the matrix inversion lemma and the observation that matrices R, T, BB^ commute, the reader 
can check easily than uq{z) = uq{z). 

In order to establish (1 145b . we remark that (11501 ) is equivalent to the linear system 



(I-D„(*)) 



a(z) — 5{z) 
a{z) — S(z) 




(156) 
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In the following, we show matrix (I — T)q(z)) is invertible for z G C+, and that the entries of its inverse 
can be bounded by terms such as Pi(|z|)P2(|Im(z)| _1 ). Proposition @] will follow immediately from 
(TT48T) and (fl49l 

We first evaluate a lower bound of det (I — Do (z)) for z G C+. For this, we introduce matrix D(z) 
defined by 

u(z) viz) 

D * = (157) 



z| v(z) u(z) 



with u,u,v,v defined by 



. . 1 <7 2 T(z)BB H T(z)" 

— Tr rar (158) 

iV |l + ar5(z)| 2 
=-^Tra 2 T(^)T(z)" (160) 

v(z) =^Tta 2 f(z)T(z) H (161) 

and define matrix D (z) as the analogue of D(z) but in which T,T, <5, 5 are replaced by R, R, a, a 
respectively. The entries of D (z) are denoted by u ,v , \z\ 2 v , u . We note that the entries of D(z) and 
D (z) are positive, and that, using the matrix inversion lemma, it is easily seen that u = u and that 
u = u . These matrices are useful because we have the following proposition. 



det(I-D(*))> /1CN2 ' 2 'L 4 (162) 



Proposition 7. There exists a strictly positive constant r] such that 

1 |Im(z 
(16) 2 (if + \z\ 2 ) 

for each z G C+ and for each N. Moreover, there exist an integer Nq and 2 polynomials Qi and Q2, 
independent of N, with positive coefficients, such that for each N > Nq, 

for each element z of the set E at defined by 

E N = {z G C+, 1 - ^2Qi(|z|)Q 2 (Im(z)- 1 ) > 0} (164) 



Finally, for each N > Nq, 



\lm(z) 



|det (I - D„(z))| > Vdet(I-D(z))Vdet(I-D'(z)) > (32)2 ^ 2 + z | 2) 4 (165) 
1/ z G E^. 
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Proof: We first establish (1 1 62b . For this, we express lm(d(z)) and lm(zS(z)) as 

Im(8{z)) = 1-Er (alm(T^))) (166) 
Im(z<5(z)) = — Tr falm^f f»)) (167) 



TV 

where for each matrix U, we define Im(U) by Im(U) = U ~P H . Writing Im(T(z)) as ^T(z)(T(z)~^- 
T(z)" 1 )T(z) H and lm(zf(z)) as ^zT(z)((zT(^))" H - (zT(z))~ l ){zT(z)) H , we get immediately that 

im(zo>)) y \ im(z<5(z)) / y w(z) I 

where w{z) and w(z) are defined by 

Mz) = iTr(^ T (z)T(^) u {z) = j 7 Tr ^f + lfM H ) (169) 

This is equivalent to 

(1 — u) Im5 = v Im(z5) + w Imz (170) 

(1 — u)lm(z5) = \z\ 2 vlm.5 + wlmz (171) 

As 5 and 5 are proportional to the Stieltjes transform of probability measures carried by R + , Im(<5) > 
0,lm(z5) > for z E C+ (see Property [5] of Lemma Q]). Therefore, (I170L 1 17 Tb imply that 1 — u = 1 — u is 
strictly positive. After some algebra, we also obtain that det (I — D) = (1 — u)(l — u) — \z\ 2 vv coincides 
with 

det (I - D) = (vw + (1 - u)w) -— (172) 

Imo 

Therefore, 

det (I - D) > (1 - u)w ^ 

Imo 

As 5{z) = acm(z), Property [3] of Lemma Q] implies that lm(5(z)) < jrf^y or equivalently that 
(Im(z)) 2 /o"c. Hence, 

det(I-D)> (1 "" Mlm(2))2 
ac 

(11701 ) implies that 

Imz w(lm(z)) 2 
l-u=l-u >w—- > 



Imz 



> 



We finally get that 



Im<5 ac 



w 2 (lm(z)) 4 

det I - D > ) .V (173) 
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In order to obtain a lower bound of w = jjTraTT H , we first remark that ±TrTT H > | jjTrT 
by the Jensen inequality. Therefore, w > ac\m\ 2 > <rc|Im(m)| 2 . lm(m(z)) can be written as 



\m\ 



R + |A-z| 2 



Im(m(z)) = Im(z) 

We recall that it is shown in |[T6l that the sequence (^n)n>o is tight. This implies that it exists 77 > 
for which UnQVi +°°[) ^ 1/2 for eacri N G IN, or equivalently for which 



Miv([0,7?]) > 1/2 (174) 



for each integer N. It is clear that 



^)> r^>_i T _ M[ o, r?] )> 



+ 



A - z\ 2 J \X-z\ 2 2(r/ 2 + \z\ 2 ) ^ JV u ' nj 4(r/ 2 + |z| 2 ^ 



Therefore, w > i^^fyy and Eq. (11731 ) gives (11621) . 

We now establish (1 1 63b - For this, we express that Im(a(z)) and lm{za{z)) as 

Im(a(z)) = — Tr (aIm(R(z))) + lm(e(z)) (175) 

lm(za(z)) = — Tr (crim(zR(z))) + Im(ze(z)) (176) 

After some algebra, we obtain that 

Im(aW> ) = D'W ( lmlaiZ)) ) + ( W ' iZ) ) lm(.) + { Im(£U) ) ,177) 
Im(za(z)) / \ lm(za(z)) I \ w (z) I \ lm(ze(z)) I 

where w (z) and w (z) are defined as w{z) and w(z) by replacing T(z), T(z),5(z),5(z) by R(z), R(z), a(z), d(z) 
respectively. This is equivalent to 

(1 — u ) Ima = f Im(za) + to Imz + Im(e(z)) (178) 
(1 — u ) Im(za) = |z| 2 v Ima + w Imz + Im(ze(z)) (179) 

These equations are of course similar to (11701 11711 ) except that the righthandsides of (11781 11791 ) are 
corrupted by the two error terms lm(e(z)) and \m(ze(z)). In order to prove (11631 ). we follow the proof 
of dl62| ) but take into account the presence of the error terms in d 1781 1179I ). As a and a are proportional 
to the Stieltjes transform of probability measures earned by R + , Im(a) > 0, Im(za) > for z £ C + . 
Therefore, (11781 ) implies that 

(1 -it') Ima > w'lm(z) - \e{z)\ (180) 

In order to determine a subset of C + on which 1 — u ' = 1 — u is strictly positive, we evaluate a 
lower bound of w (z) = -^Tr(crR(z)R(z)^). For this, we follow what preceds. We express w as 
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w = acjjTr(R(z)Ii(z) H ) and note that w > <Tc|-pTrR| 2 . As R(z) is the Stieltjes transform of a 
matrix valued measure whose mass is the matrix Im, jjTrH,(z)) is the Stieltjes transform of a probability 
measure It is shown in ||23l that jjTtH(z) — rri]Sf(z) — > for each z G C — R+. Therefore, the 
sequence — [in)n>o converges weakly forwards 0. rj > being defined by (1174I ). it thus exists an 
integer N\ for which 

Zn([0,v])>\ (181) 

for each N > N\. Using the same calculations as above, we obtain that w > ^zj^n^pp ■ Hence, using 
(fl80l and (fl48ll . we get 

(1 " u ' )Im(a) > eipT^ " ^Pid-DP^a^))- 1 ) d82) 

If we denote by E^jy the subset of C+ defined by 

Ui^+S? - ^Pid-DPilCM-))- 1 ) > (183) 
it is clear that 1 — u = 1 — u > for each N > Ni and each z G Ei jv. We note that E^jv can be 
written as 

js G C+, 1 - ^S 1 (|z|)S 2 ((Im(z))- 1 ) > o| (184) 

for some polynomials with positive coefficients. 

Using some algebra as well as the identity u = u , we get that 

det (l - D') = (v'w + (1 - u')u/) + f 'lm(ze) + (1 - u')Im(e) (185) 



Therefore, for each N > N\ and each z G Euv, we have 

I 

Ima 



/ / \ ii \\x\.z 
det ( I - D J > (1 - u )w v \ze\ 



Moreover, as £^ < ^fay , using (fT80l , we get 

(1 _ n)> -(Im(,)) 2 



ac Im(a) 

It is shown in ll23l that jjTr(E(Q(z))) — mjv(z) — > for each z G C — IR + . Therefore, the sequence 
(con — Hn)n>o converges weakly forwards where measure con is defined by ( 11021 ). rj > being defined 
by (11741 ). it thus exists an integer iVo > 2V*i for which 

un([0,v])>1 (186) 
for each N > iVo- This allows to show that Im(a) > g^ 1 ™^ for N > iVo, and that 

, W '(lm(^_8(^ + l,p 
ac aclm{z) 
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As ||R(z)|| < (Im(z))" 1 , v = iTrcr 2 RR ff verifies v < a 2 c(lm(z))~ 2 while w = -^TrcrRR^ is less 
than ac(lm(z))~ 2 . Putting all the pieces together, we obtain that 

s >Imz lm(z) 8 64(ry 2 + \z\ 2 ) 2 . . „„„ 

(1 " U )W W > (64)2(^2 + |z|2)4 " <rc (l m(z) )4 (187) 

and 

det(l-D') > , 2U - fl+ 64( f T + ' Z 'T l kW|-7f^2NI ? ( z )l ( 18§ ) 

V / (64) 2 (?7 2 + |z| 2 ) 4 \ crc(Im(z)) 4 / ' Wl (Im(z)) 2 

for N > No and for z G Ei,jv- (11881 ) can also be written as 

det ^1 — D > (6 4 ) 2 I ( " 2 ( f| z | 2) 4 (l-^M&iW*))- 1 )) 
for N > No and for z e E^jv for some polynomials with positive coefficients independent of ./V S\ and 
S 2 . We denote by E 2j tv the set 



e 2 ,at = \z e C +> ( 1 ^ 



^(M^CM*))" 1 )) > i| 



We remark that 



./v 



L € C + , 1 - ^S 1 (|z|)S 2 ((Im(z))^ 1 ) - ^W^Imiz))- 1 ) > o| C E ljiV D E 2 , 7 
We consider polynomials Qi and Q 2 defined by Qj = S, + V^S - for i = 1, 2 and define the set E_/v by 

Ejv = {a 6 C+, 1 - ^Q 1 (| Z |)Q 2 ((Im( 2; ))- 1 ) > o| 
which is included into Ei jv H E 2j at. It is clear that (11631 ) holds. 

In order to verify (11651 ). we first remark that the following inequalities hold: 

|det(I-D (*))| = |(l-u )(l-uo)-z 2 i;of)o| ( 189 ) 

> |l-uo||l-tio| - M 2 MN (190) 

> (I - \u \)(l - \u \) ~ \z\ 2 \vo\\v \ (191) 

Using the Schwartz inequality, we get that \uo\ = \uq\ < {u^^lu l 1 ^ 2 = luj 1 / 2 ^' | 1//2 , \vq\ < {v^^lv'l 1 ^ 2 , 
and \vo\ < |-y| 1//2 |-y' \ 1 / 2 . For N > No and for z G E^r, u = u < 1 and i/ = u < 1 hold. Therefore, we 
obtain that 

|det (I - D„(*))| > (1 - l^l^lu | 1/2 )(1 - \u\ 1/2 \u\ 1/2 ) - Nl 2 |«| 1/2 |«'| 1/a |«| 1/2 |«'| 1/2 (192) 
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As det(I - D(z)) = (1 - u)(l — u) - \z\ 2 vv and det(I - D'(z)) = (1 - u')(l - u) - \z\ 2 v'v are 
positive for N > Nq and for z G E^r, it is easy to check that the righthandside of (11921 ) is greater than 
(det (I - D(z))det (I - D'(z))) 1/2 for N > N 2 and for z G E N . This shows (fl65l ■ 
In order to complete the proof of (11451 ). we express a(z) — 5{z) as 

a(z) - S(z) = ^7f3p^7^ K 1 - u {z))e{z) + 2«o(z)e(z)] 

If N > N 2 , and if z G E N , flS), CM [149]), M^)| < (^g^ and |u (z)| < (recall that 

b max is defined by (O) give immediately 

Hz) - S(z)\ < ^Pxd^DPaCCImCz))- 1 ) (193) 



for some polynomials Pj, i = 1, 2 with positive coefficients. If z G C+ \ Ejy, we follow the trick of [|T8 
and 11141 . and remark that 

Irrr 

\a(z)-5(z)\ < \a(z)\ + \S(z)\ < 



lm(z) 

If z G C+ \ E N , 2 < 7 ^Q 1 (|z|)Q 2 ((Im(z))- 1 ) so that 

K*) - < i ^y^2Qi(kl)Q2((M^))- 1 ) 

Therefore, for N > No, and for each z G C+, 

|a(*)-*(*)| < ^ (Pi(NI)P 2 ((Im(z))- 1 ) + ^QidzDQaCM*)) -1 )) < j^Ckl+^QCM*)) -1 ) 

where k is an integer, C is a positive constant and Q is a positive coefficients polynomial. Proposition 
S follows directly from the identity a(z) - S(z) = ac (E(^TrQ(z)) - ^TrT(z)). 

Appendix F 
Proof of (fT6i 

We first show that for each z G C+, vl^(Q n (z)— TVz)) v iV converges towards on a set of probability 
1 which, in principle, depends on z. In order to obtain the almost sure convergence towards for each 
z G C — IR_|_, we use a standard argument based on Montel's theorem. 

We first write 

u$ (Q N (z) - TV(*)) vtv = u$ (Qn(z) - E(Q N (z))) v N + u$ (E(Q N (z)) - T N (z)) v N (194) 
We study the second term of the righthandside of dl94| ) and write 

u# (E(Q N (z)) - T N {z)) VJV = u$ (E(Q N (z)) - T N (z)) V7V + u$ (R N (z)) - T N {z)) v N 
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where we recall that matrix Rjv(z) is defined by (I108I ). (11451 ) implies that oln(z) — 5n(z) and oin(z) — 
5n(z) converge towards (cw, <$jv, &N, $N are defined by (11001 l98l 1101 L l99l respectively) . Using the 
identity K N (z)-T N (z) = K N {z) (T^(z) - R^ 1 ^)) Tat(z) allows to express u% (Rat(z)) - T N (z)) vjv 
as a linear combination of a^iz) — 6n(z) and &n(z) — 5n(z). As ||Rjv(^))|| < |Im(2;)| _1 , ||Tjv(^))|| < 
|Im(z)| _1 , the coefficients of this linear combination remain bounded when N — > +oo. This shows that 
(Rjv(z)) — Tjv(z)) vat converges towards 0. 

In order to study (E(Qjv(z)) — Rjv(z)) Vat, we use relation (|121l) . Using the Nash-Poincare in- 
equality, it is easy to check that u^Riv(z) Aat(z)vtv — > 0. (11401 ) implies moreover that -^Tr An(z) — >■ 0. 
(11211) thus shows that (E(Qat(z)) - K N {z)) v N ->■ 0. 

It remains to prove that xat(z) = (Qat(z) — ^(Qn(z)))^n converges towards almost surely. 
For this, it is sufficient to show that 

E|xaKz)| 4 < ^ (195) 

where C(z) does not depend on N. We express E|xjv(z)| as 

E\x N (z)\ A = \ E(x N (z) 2 )\ 2 + Var(x N (z)) 2 

We remark that |E(xat(2:) 2 )| 2 < (E\xn(z)\ 2 ) 2 . Moreover, E(xn(z)) = implies that E|xat(2:)| 2 = 
Var(xAr(z)). Therefore, 

E\x N (z)\ 4 < (Vw(x N (z))) 2 + Var [(x N (z)Y 

Using the Nash-Poincare inequality, it is easy to show that Var(x/v(z)) < and that Var (xn(z) 2 ) < 
This establishes ( 11951 ) and that (Qat(z) — Tat(z)) converges towards on a set of probability 
1 depending on z. 

In order to prove the almost sure convergence for each z G C — IR+, we use the following standard 
argument. We consider a countable subset Z c C C+ having an accumulation point. On a set O of 
probability 1 , (Qn{z) — TV(z)) — > for each z G Z c . We fix a realization of the set $7. We 
denote by dn{z) the function un(z) = (Qn{z) — Tn(z))vn. Functions z — > u^Qn(z)vn and 
z — > u^T n(z)vn are Stieltjes transforms of bounded measures carried by K + . Therefore, function 
is analytic on C — R + , and for each compact subset K. of C — R+, it holds that 

C 



|yjv(^)| < 



dist(/C, R+) 

for some constant C (this is a trivial generalization of (O to the Stieltjes transform of a non necessarily 
positive bounded measure carried by R + ). Montel's theorem dl24l0 thus implies that it exists a subsequence 
y^,(Ar) extracted from yjv which converges uniformly on each compact subset of C — R + towards a certain 
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function y* which is analytic on C — R+. However, y*(z) = for each z £ Z c , thus showing that is 
identically on C — IR+. The limit of each converging subsequence extracted from y^ is thus identically 
0. We thus obtain that the whole sequence yjy converges uniformly towards on each compact subset 
of C — R+. Therefore, for each realization of the probability 1 set Q, we have shown that 

u%{Qn(z) -T n (z))v n ->0 
for each z £ C — IR+. This completes the proof of (fl6l) . 



Appendix G 
Proof of Lemma 0] 



An elementary study of function x — > rhj^(x) shows that ojk € ^jt > -^i+i 



that Cj^P > X^. Therefore, by Theorem [4] we only need to prove that wj^-if < arm ost surely for 
all sufficiently large N. 



,(AT)" 



M - 1 and 



Consider the contour C defined in Proposition [5] Noting that C encloses {0} on the complex plane and 
that Indc(O) = 1, we can write 



— / A -1 dA 
2vri J c+ 

'■t 



1 
2^i 



w' N (x) 



dx 

2vri 



t+1 w' N {x) 



dx 



(196) 



(197) 



f t - \w N (x) J 2iriJ t - w N (x) 

where the notation C + means that the contour C is counterclockwise oriented. Since functions h i-> 
wn(x + ih) and h \-> w' N (x + ih) are continuous at h = for all x E]t^,tf[ (except for the points 
x G |scj^ ,x^^ + |), Lemma [3] together with the Dominated Convergence Theorem imply that 



1 



lim 

2/4-0 



lim 

1/4-0 



1 

2vri 

1 
2vri 



w' N (x + iy) \* 
w N (x + iy) 



dx 



J_ Z"' 1 w' N (x + iy) 
2-KiJt- w N {x + iy) 



dx 



Wn{z) 27T 



-y U)N(ti — ih) 2tt 



y w ' N (tf + ih) 



dh 



(198) 



(199) 



y WN(t{ + ih) 
counterclockwise oriented. The function 



where dlZ^ denotes the contour of the rectangle defined in 
h i — y Z)(x+ih) * s a continuous function on the compact set [—y, y] for x = t± or tf, and therefore the 
two last integrals vanish as y J, 0, so that we can write 



1 = lim 



i_ r w' N (z) 

yio 2vri J dn + w N (z) 



dz. 
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Since the function ^|^y is holomorphic on C\[rc^ ,x± + ], the last integral does not depend on the 
value of y > 0, and thus we can drop the limit, i.e. 

1 1 w 'n( z ) 



27ri Jam w N {z) 

This identity will be key in order to prove that Cjm-k < tf almost surely for all sufficiently large N. 



Before going further into the proof of this result, let us first examine the function wn (z) defined by 
64l ) when z G R. The following result follows from elementary analysis: 




Figure 13. Typical representation of wn (x) as a function of x for M = 3 (we drop the dependence with N from all quantities 
for clarity). 



Lemma 9. The function w n defined on IR by 

wn{x) = x (l + o- 2 CNrhN(x)Y 
satisfies ( see further Figure |7?1 ) 

lim wn{x) = +oo, 
lim wn(x) = +oo, 

x— >+oo 

September 30, 2011 



cr 2 (l - cat) (l + a 2 CNrriN(x)) 

lim wn{x) = +00 (201) 

ztAfc 

lim wn(x) = —00. (202) 

X— > — 00 
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Moreover, wn(x) = is a polynomial equation with degree 2M + 1 with the following zeros: 

• One zero in 0, A^ , denoted as z$. 

• Two zeros in each interval \\^\ A^Y^ , denoted as Cj^\ z^\ k = 1 . . . M — 1. 



• Two zeros in X^, +°° . denoted as Cj^K z^ 
Furthermore, we have 

< i N) < \[ N) < u[ N) < z[ N) < Af ) < . . . 

• • • < A k <UJ k <z k < \+i < • • • < A M <UJ M < Z M ■ 

Now, the function z — > wn(z), denned on C, is holomorphic everywhere except at poles (of order 2) 
A^ ' ' Moreover, function z —> ^"^j is holomorphic everywhere except at the zeros of wn 

and at the sample eigenvalues A^ , . . . , Xffl. 

Figure [T4l gives an schematic representation of the positions of the zeros and poles of wn(x) in terms 
of the contour dlZ y . Observe that, for sufficiently high N, Theorem |4] ensures that |a^, . . . , A^l^l 
will be inside dlZ y , whereas the rest of the sample eigenvalues will be outside. Given the position of the 
zeros , established in Lemma |9l we see that the position of the sample eigenvalues determines 
that the zeros jcD^, z^\ k = 1 . . . M — K — l| will also be inside dlZ y for all N sufficiently high. 
Furthermore, the remaining zeros will be outside dlZy, except for the zeros Zq N \ <^m-k anc ' z< m-k> f° r 
which we can not state anything. In what follows, we will see that these three zeros are in fact located 
inside dlZ y with probability one for all large N, which will conclude the proof of Lemma |4] As a first 
step, we introduce an intermediate result that establishes that none of these zeros can converge to a the 
boundary point of d1Z y when N — > +oo. 

Lemma 10. For all N large enough, Zq 7= t^, <^^jl^ 7= tf and z^~}_ K 7^ tf. 

Proof: We will just establish that Cj^_ k 7^ tf and z^_ K 7^ tf, since the proof that Zq 7^ t ± is 
quite similar. For this, we prove the following: 

inf inf \w N (x)\ > (203) 
lim sup \wn(x) — wn(x)\ = a.s (204) 

If (12031 12041 ) hold true, it is clear that almost surely, it exists N± G IN for which 

inf inf \wn(x)\ >0a.s. (205) 

N>N lxe [ t + >t -]' 



(N) JN) 
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8TU 



r, 



z 



(N) AN) r (N) t JN 



2 ^2 



<? *^ 



X • • 



,(JV) 



w M-/f-l 2 M-A'-1 



-X- 



-X- 



;jv) v» 



-X- 



^ A (JV) 



Figure 14. Schematic representation of the position of the zeros (circles) and poles (crosses) of the function wn(z) on the 
region enclosed by dTZ y . 



a property which implies that Gj^P-K ^ an ^ ^m-k ^ f° r N > Ni. 

In order to prove (12031) . we note that Assumptions Q] and [2] imply the existence of e > such that 

wn{x) > if x e — e,t^ + e] and iV > N$. Now, we write wn(z) as 

wn(z) = z(l + (j5n(z))(1 + ct5n(z)) = z(l + a 2 CNm N (z))(l + a 2 CNrriN(z) - ——^ ° N ^ ) ( 206 ) 



where we recall that 5n and 5n are defined by (1981 ) and (1991 ) respectively. It has been mentioned in Ap- 
pendix IE] that function z — > — , \ , \ \ = — m — 5-^ tty coincides with the Stielties transform of a 

v 1 — 1 z(l+cr(5jv(2)) z(l+cr 2 c N m N (z)) ■> 

probability measure carried by K + . We denote by 77V this measure. As wj^(x) > if x G [tf — e, £J + e] , 
function z — > — z ^ + Js N ( z )) * s ana lyti c on C+ U C-U]tf — e, t% + e[ and is real-valued on — e, ij + e]. 
The support of measure 77V is thus included into IR + — ]tf — e, t% + e[. Therefore, Property [9] of Lemma 
Q] implies that 

+ a 2 c N m N {x))[' 1 < - (207) 
for each x E [it , tn]- It can also be shown that z — > — ; ; - = — r— — ~ j-^ — ™ r coincides 

L 1 ' 2 J z(1+o-5jv(z)) 2(l+cr 2 civmjv(z))-cr 2 (l-cjv) 

with the Stieltjes transform of a probability measure carried by IR + . Using the same approach as above, 
we obtain that 

\x(l + o 2 c N m N (x)) - a 2 (l - cn)^ 1 < - (208) 



c 



for each i£ [i+,t 2 ]. This, in turn, implies ( I203I ). 
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In order to establish (12041 ). we note that it is sufficient to establish that 



lim sup |mjv(x) — ttin(x)\ = a.s (209) 

Ar ^+°°xG[t+,t 2 -] 

Theorem |4] implies the existence of e > for which, almost surely, function z — > m^{z) is analytic on 
C + U C-U]tf — e, + e[ for N > N\ where N\ > N is a certain integer. Eq. (0 implies that for each 
compact subset K of C+ U C-U]tf — e, + e[, there exists a constant C(/C) for which almost surely 
sup N>Ni supz^fc \rh,N(z)\ < C(JC). For the same reasons, it holds that sup N>Ni sup zeK: |mjv(^)| < 
C(JC). Montel's Theorem dl24T ) thus implies that it exists a subsequence m^ny) — VfiMm extracted from 
(mjv — tun)n > n 1 which converges uniformly on each compact subset of C+ U C-U]i^ — e, ^ + e[ 
forwards a function p*(z), analytic on C + U C-Ujt] 1- — e, t% + e[. Proposition Q] implies that almost surely, 
rhjv(z) — mjv(^) — >■ for each 2 G C\R+. This implies that p*(^) is identically zero. As the limit of each 
convergent subsequence extracted from ihjy — is 0, the whole sequence (mjv — tun) tv>at 1 converges 
uniformly forwards on each compact subset of C+ U C_U]t^ — e, t% + e[. This, of course, implies (1209b - 
This completes the proof of Lemma [TOl 

■ 

Using the same arguments as above, it is easy to show that there exists N2 G IN such that infjv>jv 2 inf z e37£ I^aK- 2 )! > 
and such that, almost surely, infjv>iv 2 inf^eST?.,, |wjv(^)| > 0. It also holds that sup Ar>A r 2 sup 26(97 ^ |?«^-(«)| < 
+00 and sup Ar>A r 2 sup 26 g^ |it^y(.s)| < +00 almost surely. Since almost surely the function — 
converges to for each z G <97£ y , the Dominated Convergence Theorem ensures that, with probability 



one, 



1 / W N (z) _ w' N (z) 
27ri J&R+ [wn(z) w n (z) 



dz 



iv->+oo 



-> 



Now, according to Lemma [TOl z^ ^ t 1 ,co M _ K 7^ ^\i^m-K ^ ^1 w i tn probability one for all large 

on contour &R y . 

1 I w' N {z) 



N. Hence, it is possible to use the argument principle to function w ~ on contour d1Z y . More precisely, 



f V^^d z = caxd{z€:w N (z) = 0}-2(M-K) 

27rl J&R+ W N [Z) 

and since the previous integral is an integer, using (I2001 ). we finally have with probability one for N 
large enough 

2(M - K) + 1 = card {z G 1Z y : w N (z) = 0} . 

We already know that z[ N \. . . ,z^^_ K _ 1 and ,. . . ,oj3_ k _ x , which are zeros of w^{z), belong to 
TZ y . Since the total number of zeros is 2M + 1, 3 other zeros of wn(z) belong to lZ y with probability one 
for ./V large enough. However, all the zeros of wn(z) are real- valued, which implies that the 3 additional 
zeros necessarily include £o^_ K . This concludes the proof Lemma [4] 
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Appendix H 
Proof of (1671) and 

We first establish (|67T ). For this, we recall that TV (2) is the Stieltjes transform of a positive matrix 
valued measure IV with mass In- Therefore, function z — > h^TN{z)hN coincides with the Stieltjes 
transform of the positive measure bjylVbjv- This measure is clearly absolutely continuous w.r.t. measure 
Tr(IV), or equivalently w.r.t. measure /j,n = -pTr(IV). The support of bjylVbjv is thus contained 
into Sn- Therefore, it holds that 

\b N T N (z)b N \ < 



dist(z, <Sjv) 

(see (|9]). We have already mentioned in Appendix |E] and in Appendix iGl that function z — > (—^(1 + a 2 CNmN(z))) 1 
is the Stieltjes transform of a probability measure carried by R + . This function is moreover analytic in 
C — Sn because 1 + a 2 CNmN(z) / on C - Sn (see Property [6] of Proposition Q]), a property which 
implies that the support of its associated measure is included into Sn- Therefore, we have 



-z(l + a 2 CNmN{z))\ 1 < 



dist(2;, Sn) 



or equivalently 



1 + a CNmN{z))\ < 



dist(z, Sn) 

Assumptions (Q~|) and (0 imply that iiuV>7v dist{d!Z y , S n) > 0. We thus obtain that 

\h N T N (z)h N \ 
sup sup " —— < +00 

n>n zed-Ry |1 + a- z c N m N (z))\ 

Using again that kdV>Ar dist(dTly, Sn) > 0, it can be checked that sup N >N su Vz£dH y \ w n( z )\ < +°°- 
This in turn establishes (l67l) . 

In order to prove (I68T ). we recall that mjv(z) is the Stieltjes transform of the probability measure [in = 
~M Sfcli ~~ )• Assumptions (Q} and © imply it exists Nq £ N such that the distance between 
cflZy and the support of /V is lower bounded by a strictly positive term independent of N > Nq. It is 
easily seen that z — > h^QN(z)hN is the Stieltjes transform of measure jj Y^k=i |bjv^i | 2< ^(^ ~~ ^fc^)- 
The support of this measure is included into {A^ , . . . , A^}. Using Q as above, we deduce from this 
that 

sup sup b^QN(z)hN < +00 

N>N zedTZy 

The same arguments can be used to show that sup N>No sup z£ Q-ji y \ w N (z)\ < +00. 

Finally, using Property [6] of LemmaQ] it is easily seen that function z — > (— z(l + a 2 CNrhN{z))) 1 is 
the Stieltjes transform of a probability measure. Its support is included into the set {A^ , ■ ■ ■ , Xffl , tu^ , . . . , tbffi ) 
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Moreover, in the statement of Lemma HI tj~ and tf can be replaced by + ei and — e\ where ei 
is chosen in such a way that t\ + ei < infjv>jVo < supjy^jy < tf — €\. Therefore, the 

distance between d!Z y and {A^ , . . . , X^\lo[ N \ . . . } is lower bounded by a strictly positive term 
independent of N > Nq. This implies that 

sup sup |l + (T 2 CNrriN{z)\ 1 < +00 

7V>7V z£dTl v 



This completes the proof of 

Appendix I 
Proof of Lemma [5] 

We first write the equation in uj, 1 + a 2 cmN(uj) = as 
and by multiplying the left hand side by Y\iL\ (^j ~ we define a new polynomial Q(uj), by 

2 MM M 

«")=^En(M + n(*H- 

As the monic polynomial function Q has M roots at d>i, . . . we can write 

M 

qm = n - ^) 

1=1 

Therefore, 

M 9 M 

O C N 



M 

1=1 1=1 
l^k 



Q(A fc )=n =-rfn ^-^ ( 2i D 



which will be useful later on. Let us now consider the derivative of Q given by 

M M M M 9 M M M 

cr c N 

M 

3=1 =1 j=l^J m=l J=l i=l 

Z=l l^mj^m,l 

Evaluating again this function at point we obtain 

M M M 2 M M 

Q'cx k ) = -En (*<-**)= -n (*< - **) - n - **) 

j=i 1=1 1=1 1=1 j=i 

l¥=3 l^k l^k j^k,l 

or, dividing both sides by the first term on the right hand side of the equation, 

ttM 



E,=irL=i [&i - Afe) 2 w 1 

A/A 
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Going back to equation (121 lb . one can also write 

xlL n| (** - ^) _ a2cN tsu n| (*, - _ a2cjv m 1 

Consequently, we see that we can write 

o 2 A/ 1 2 1 2 M i 

2a cn st~^ 1 °" c iV 1 c Civ 1 

+ M h^-^ = M "k-\ k + ^ r h 6j i-" x k 

or, reorganizing the terms of this expression in a convenient way, 



2 1 2 M 1 2 M 1 2 M i 

M X k -u> k M ^A,-A fc " M f^un-Xk M ^ X t - X ' 

l^k l^k l^k 

But from the equation in to (1210b . we obtain 

2 i 2 M 1 

a CM 1 £7 C/V 1 

H 1 -> = 

M X k -co k M ^A z -% 

l+k 

and by inserting this expression into (12151 ). we finally get the expression in the lemma. 



k 
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